SwiftFileBackend Class Reference

Class for an OpenStack Swift (or Ceph RGW) based file backend. More...

Inheritance diagram for SwiftFileBackend:

Collaboration diagram for SwiftFileBackend:

List of all members.

Public Member Functions
	__construct (array $config)
	getDirectoryListInternal ($fullCont, $dir, array $params)
	getDirListPageInternal ($fullCont, $dir, &$after, $limit, array $params)
	Do not call this function outside of SwiftFileBackendFileList.
	getFileHttpUrl (array $params)
	getFileListInternal ($fullCont, $dir, array $params)
	getFileListPageInternal ($fullCont, $dir, &$after, $limit, array $params)
	Do not call this function outside of SwiftFileBackendFileList.
	isPathUsableInternal ($storagePath)
	Check if a file can be created or changed at a given storage path.
	purgeCDNCache (array $objects)
	Purge the CDN cache of affected objects if CDN caching is enabled.
Protected Member Functions
	_getResponseCopy (CF_Async_Op $cfOp, Status $status, array $params)
	_getResponseCreate (CF_Async_Op $cfOp, Status $status, array $params)
	_getResponseDelete (CF_Async_Op $cfOp, Status $status, array $params)
	_getResponseMove (CF_Async_Op $cfOp, Status $status, array $params)
	_getResponseStore (CF_Async_Op $cfOp, Status $status, array $params)
	addMissingMetadata (CF_Object $obj, $path)
	Fill in any missing object metadata and save it to Swift.
	closeConnection ()
	Close the connection to the Swift proxy.
	createContainer ($container)
	Create a Swift container.
	deleteContainer ($container)
	Delete a Swift container.
	directoriesAreVirtual ()
	Is this a key/value store where directories are just virtual? Virtual directories exists in so much as files exists that are prefixed with the directory path followed by a forward slash.
	doCleanInternal ($fullCont, $dir, array $params)
	doCopyInternal (array $params)
	doCreateInternal (array $params)
	doDeleteInternal (array $params)
	doDescribeInternal (array $params)
	doDirectoryExists ($fullCont, $dir, array $params)
	doExecuteOpHandlesInternal (array $fileOpHandles)
	doGetFileContentsMulti (array $params)
	doGetFileSha1base36 (array $params)
	doGetFileStat (array $params)
	doGetLocalCopyMulti (array $params)
	doMoveInternal (array $params)
	doPrepareInternal ($fullCont, $dir, array $params)
	doPrimeContainerCache (array $containerInfo)
	Fill the backend-specific process cache given an array of resolved container names and their corresponding cached info.
	doPublishInternal ($fullCont, $dir, array $params)
	doSecureInternal ($fullCont, $dir, array $params)
	doStoreInternal (array $params)
	doStreamFile (array $params)
	getConnection ()
	Get an authenticated connection handle to the Swift proxy.
	getContainer ($container, $bypassCache=false)
	Get a Swift container object, possibly from process cache.
	getParentDir ($path)
	handleException (Exception $e, $status, $func, array $params)
	Log an unexpected exception for this backend.
	headersFromParams (array $params)
	Get headers to send to Swift when reading a file based on a FileBackend params array, e.g.
	resolveContainerPath ($container, $relStoragePath)
	sanitizeHdrs (array $headers)
	setContainerAccess (CF_Container $contObj, array $readGrps, array $writeGrps)
	Set read/write permissions for a Swift container.
	truncDisp ($disposition)
Protected Attributes
CF_Authentication	$auth
	*
	$authTTL
CF_Connection	$conn
	*
ProcessCacheLRU	$connContainerCache
	*
	$connErrorTime = 0
CloudFilesException	$connException
	*
	$rgwS3AccessKey
	$rgwS3SecretKey
	$sessionStarted = 0
BagOStuff	$srvCache
	*
	$swiftAnonUser
	$swiftCDNExpiry
	$swiftCDNPurgable
	$swiftTempUrlKey
	$swiftUseCDN
Private Member Functions
	getCredsCacheKey ($username)
	Get the cache key for a container.
	loadObjectListing (array $params, $dir, array $cfObjects)
	Load a list of objects that belong under $dir into stat cache and return a list of the names of the objects in the same order.

---

Detailed Description

Class for an OpenStack Swift (or Ceph RGW) based file backend.

This requires the SwiftCloudFiles MediaWiki extension, which includes the php-cloudfiles library (https://github.com/rackspace/php-cloudfiles). php-cloudfiles requires the curl, fileinfo, and mb_string PHP extensions.

Status messages should avoid mentioning the Swift account name. Likewise, error suppression should be used to avoid path disclosure.

Since:

1.19

Definition at line 39 of file SwiftFileBackend.php.

---

Constructor & Destructor Documentation

SwiftFileBackend::__construct

(

array $

config

)

See also:

FileBackendStore::__construct() Additional $config params include:

• swiftAuthUrl : Swift authentication server URL
• swiftUser : Swift user used by MediaWiki (account:username)
• swiftKey : Swift authentication key for the above user
• swiftAuthTTL : Swift authentication TTL (seconds)
• swiftTempUrlKey : Swift "X-Account-Meta-Temp-URL-Key" value on the account. Do not set this until it has been set in the backend.
• swiftAnonUser : Swift user used for end-user requests (account:username). If set, then views of public containers are assumed to go through this user. If not set, then public containers are accessible to unauthenticated requests via ".r:*" in the ACL.
• swiftUseCDN : Whether a Cloud Files Content Delivery Network is set up
• swiftCDNExpiry : How long (in seconds) to store content in the CDN. If files may likely change, this should probably not exceed a few days. For example, deletions may take this long to apply. If object purging is enabled, however, this is not an issue.
• swiftCDNPurgable : Whether object purge requests are allowed by the CDN.
• shardViaHashLevels : Map of container names to sharding config with:
  • base : base of hash characters, 16 or 36
  • levels : the number of hash levels (and digits)
  • repeat : hash subdirectories are prefixed with all the parent hash directory names (e.g. "a/ab/abc")
• cacheAuthInfo : Whether to cache authentication tokens in APC, XCache, ect. If those are not available, then the main cache will be used. This is probably insecure in shared hosting environments.
• rgwS3AccessKey : Ragos Gateway S3 "access key" value on the account. Do not set this until it has been set in the backend. This is used for generating expiring pre-authenticated URLs. Only use this when using rgw and to work around http://tracker.newdream.net/issues/3454.
• rgwS3SecretKey : Ragos Gateway S3 "secret key" value on the account. Do not set this until it has been set in the backend. This is used for generating expiring pre-authenticated URLs. Only use this when using rgw and to work around http://tracker.newdream.net/issues/3454.

Reimplemented from FileBackendStore.

Definition at line 100 of file SwiftFileBackend.php.

---

Member Function Documentation

SwiftFileBackend::_getResponseCopy	(	CF_Async_Op $	cfOp,
		Status $	status,
		array $	params
	)		[protected]

See also:

SwiftFileBackend::doExecuteOpHandlesInternal()

Definition at line 432 of file SwiftFileBackend.php.

SwiftFileBackend::_getResponseCreate	(	CF_Async_Op $	cfOp,
		Status $	status,
		array $	params
	)		[protected]

See also:

SwiftFileBackend::doExecuteOpHandlesInternal()

Definition at line 277 of file SwiftFileBackend.php.

SwiftFileBackend::_getResponseDelete	(	CF_Async_Op $	cfOp,
		Status $	status,
		array $	params
	)		[protected]

See also:

SwiftFileBackend::doExecuteOpHandlesInternal()

Definition at line 552 of file SwiftFileBackend.php.

SwiftFileBackend::_getResponseMove	(	CF_Async_Op $	cfOp,
		Status $	status,
		array $	params
	)		[protected]

See also:

SwiftFileBackend::doExecuteOpHandlesInternal()

Definition at line 504 of file SwiftFileBackend.php.

SwiftFileBackend::_getResponseStore	(	CF_Async_Op $	cfOp,
		Status $	status,
		array $	params
	)		[protected]

See also:

SwiftFileBackend::doExecuteOpHandlesInternal()

Definition at line 361 of file SwiftFileBackend.php.

SwiftFileBackend::addMissingMetadata	(	CF_Object $	obj,
		$	path
	)		[protected]

Fill in any missing object metadata and save it to Swift.

Parameters:

CF_Object	$obj
string	$path	Storage path to object

Returns:

bool Success

Exceptions:

Exception

cloudfiles exceptions

Definition at line 780 of file SwiftFileBackend.php.

SwiftFileBackend::closeConnection

(

)

[protected]

Close the connection to the Swift proxy.

Returns:

void

Definition at line 1401 of file SwiftFileBackend.php.

SwiftFileBackend::createContainer

(

$

container

)

[protected]

Create a Swift container.

Parameters:

string

$container

Container name

Returns:

CF_Container

Exceptions:

CloudFilesException

Definition at line 1456 of file SwiftFileBackend.php.

SwiftFileBackend::deleteContainer

(

$

container

)

[protected]

Delete a Swift container.

Parameters:

string

$container

Container name

Returns:

void

Exceptions:

CloudFilesException

Definition at line 1470 of file SwiftFileBackend.php.

SwiftFileBackend::directoriesAreVirtual

(

)

[protected]

Is this a key/value store where directories are just virtual? Virtual directories exists in so much as files exists that are prefixed with the directory path followed by a forward slash.

Returns:

bool

Reimplemented from FileBackendStore.

Definition at line 1238 of file SwiftFileBackend.php.

SwiftFileBackend::doCleanInternal	(	$	container,
		$	dir,
		array $	params
	)		[protected]

See also:

FileBackendStore::doClean()

Returns:

Status

Reimplemented from FileBackendStore.

Definition at line 710 of file SwiftFileBackend.php.

SwiftFileBackend::doCopyInternal

(

array $

params

)

[protected]

See also:

FileBackendStore::copyInternal()

Returns:

Status

Reimplemented from FileBackendStore.

Definition at line 371 of file SwiftFileBackend.php.

SwiftFileBackend::doCreateInternal

(

array $

params

)

[protected]

See also:

FileBackendStore::createInternal()

Returns:

Status

Reimplemented from FileBackendStore.

Definition at line 217 of file SwiftFileBackend.php.

SwiftFileBackend::doDeleteInternal

(

array $

params

)

[protected]

See also:

FileBackendStore::deleteInternal()

Returns:

Status

Reimplemented from FileBackendStore.

Definition at line 512 of file SwiftFileBackend.php.

SwiftFileBackend::doDescribeInternal

(

array $

params

)

[protected]

See also:

FileBackendStore::describeInternal()

Returns:

Status

Reimplemented from FileBackendStore.

Definition at line 564 of file SwiftFileBackend.php.

SwiftFileBackend::doDirectoryExists	(	$	fullCont,
		$	dir,
		array $	params
	)		[protected]

See also:

FileBackendStore::doDirectoryExists()

Returns:

bool|null

Reimplemented from FileBackendStore.

Definition at line 878 of file SwiftFileBackend.php.

References $dir, array(), getContainer(), and handleException().

SwiftFileBackend::doExecuteOpHandlesInternal

(

array $

fileOpHandles

)

[protected]

See also:

FileBackendStore::executeOpHandlesInternal()

Parameters:

array

$fileOpHandles

Exceptions:

MWException

Returns:

Array List of corresponding Status objects

Reimplemented from FileBackendStore.

Definition at line 1258 of file SwiftFileBackend.php.

SwiftFileBackend::doGetFileContentsMulti

(

array $

params

)

[protected]

See also:

FileBackendStore::getFileContentsMulti()

Returns:

Array

Reimplemented from FileBackendStore.

Definition at line 806 of file SwiftFileBackend.php.

SwiftFileBackend::doGetFileSha1base36

(

array $

params

)

[protected]

Definition at line 1079 of file SwiftFileBackend.php.

SwiftFileBackend::doGetFileStat

(

array $

params

)

[protected]

See also:

FileBackendStore::getFileStat()

Reimplemented from FileBackendStore.

Definition at line 745 of file SwiftFileBackend.php.

SwiftFileBackend::doGetLocalCopyMulti

(

array $

params

)

[protected]

See also:

FileBackendStore::getLocalCopyMulti()

Returns:

Array

Reimplemented from FileBackendStore.

Definition at line 1124 of file SwiftFileBackend.php.

SwiftFileBackend::doMoveInternal

(

array $

params

)

[protected]

See also:

FileBackendStore::moveInternal()

Returns:

Status

Reimplemented from FileBackendStore.

Definition at line 440 of file SwiftFileBackend.php.

SwiftFileBackend::doPrepareInternal	(	$	container,
		$	dir,
		array $	params
	)		[protected]

See also:

FileBackendStore::doPrepare()

Returns:

Status

Reimplemented from FileBackendStore.

Definition at line 597 of file SwiftFileBackend.php.

SwiftFileBackend::doPrimeContainerCache

(

array $

containerInfo

)

[protected]

Fill the backend-specific process cache given an array of resolved container names and their corresponding cached info.

Only containers that actually exist should appear in the map.

Parameters:

array

$containerInfo

Map of resolved container names to cached info

Returns:

void

Reimplemented from FileBackendStore.

Definition at line 1476 of file SwiftFileBackend.php.

SwiftFileBackend::doPublishInternal	(	$	fullCont,
		$	dir,
		array $	params
	)		[protected]

See also:

FileBackendStore::doPublishInternal()

Returns:

Status

Reimplemented from FileBackendStore.

Definition at line 674 of file SwiftFileBackend.php.

References array(), getContainer(), handleException(), Status\merge(), Status\newGood(), and setContainerAccess().

SwiftFileBackend::doSecureInternal	(	$	fullCont,
		$	dir,
		array $	params
	)		[protected]

See also:

FileBackendStore::doSecureInternal()

Returns:

Status

Reimplemented from FileBackendStore.

Definition at line 639 of file SwiftFileBackend.php.

References array(), empty, getContainer(), handleException(), Status\merge(), Status\newGood(), and setContainerAccess().

SwiftFileBackend::doStoreInternal

(

array $

params

)

[protected]

See also:

FileBackendStore::storeInternal()

Returns:

Status

Reimplemented from FileBackendStore.

Definition at line 285 of file SwiftFileBackend.php.

SwiftFileBackend::doStreamFile

(

array $

params

)

[protected]

See also:

FileBackendStore::streamFile()

Returns:

Status

Reimplemented from FileBackendStore.

Definition at line 1093 of file SwiftFileBackend.php.

SwiftFileBackend::getConnection

(

)

[protected]

Get an authenticated connection handle to the Swift proxy.

Exceptions:

CloudFilesException
CloudFilesException|Exception

Returns:

CF_Connection|bool False on failure

Definition at line 1355 of file SwiftFileBackend.php.

SwiftFileBackend::getContainer	(	$	container,
		$	bypassCache = false
	)		[protected]

Get a Swift container object, possibly from process cache.

Use $reCache if the file count or byte count is needed.

Parameters:

string	$container	Container name
bool	$bypassCache	Bypass all caches and load from Swift

Returns:

CF_Container

Exceptions:

CloudFilesException

Definition at line 1429 of file SwiftFileBackend.php.

Referenced by doDirectoryExists(), doPublishInternal(), and doSecureInternal().

SwiftFileBackend::getCredsCacheKey

(

$

username

)

[private]

Get the cache key for a container.

Parameters:

string

$username

Returns:

string

Definition at line 1416 of file SwiftFileBackend.php.

SwiftFileBackend::getDirectoryListInternal	(	$	fullCont,
		$	dir,
		array $	params
	)

See also:

FileBackendStore::getDirectoryListInternal()

Returns:

SwiftFileBackendDirList

Reimplemented from FileBackendStore.

Definition at line 897 of file SwiftFileBackend.php.

References $dir.

SwiftFileBackend::getDirListPageInternal	(	$	fullCont,
		$	dir,
		&$	after,
		$	limit,
		array $	params
	)

Do not call this function outside of SwiftFileBackendFileList.

Parameters:

string	$fullCont	Resolved container name
string	$dir	Resolved storage directory with no trailing slash
string | null	$after	Storage path of file to list items after
integer	$limit	Max number of items to list
array	$params	Parameters for getDirectoryList()

Returns:

Array List of resolved paths of directories directly under $dir

Exceptions:

FileBackendError

Definition at line 920 of file SwiftFileBackend.php.

SwiftFileBackend::getFileHttpUrl

(

array $

params

)

See also:

FileBackend::getFileHttpUrl()

Returns:

string|null

Reimplemented from FileBackendStore.

Definition at line 1193 of file SwiftFileBackend.php.

SwiftFileBackend::getFileListInternal	(	$	fullCont,
		$	dir,
		array $	params
	)

See also:

FileBackendStore::getFileListInternal()

Returns:

SwiftFileBackendFileList

Reimplemented from FileBackendStore.

Definition at line 905 of file SwiftFileBackend.php.

References $dir.

SwiftFileBackend::getFileListPageInternal	(	$	fullCont,
		$	dir,
		&$	after,
		$	limit,
		array $	params
	)

Do not call this function outside of SwiftFileBackendFileList.

Parameters:

string	$fullCont	Resolved container name
string	$dir	Resolved storage directory with no trailing slash
string | null	$after	Storage path of file to list items after
integer	$limit	Max number of items to list
array	$params	Parameters for getDirectoryList()

Returns:

Array List of resolved paths of files under $dir

Exceptions:

FileBackendError

Definition at line 995 of file SwiftFileBackend.php.

SwiftFileBackend::getParentDir

(

$

path

)

[protected]

Definition at line 980 of file SwiftFileBackend.php.

SwiftFileBackend::handleException	(	Exception $	e,
		$	status,
		$	func,
		array $	params
	)		[protected]

Log an unexpected exception for this backend.

This also sets the Status object to have a fatal error.

Parameters:

Exception	$e
Status	$status,|null
string	$func
array	$params

Returns:

void

Definition at line 1499 of file SwiftFileBackend.php.

Referenced by doDirectoryExists(), doPublishInternal(), and doSecureInternal().

SwiftFileBackend::headersFromParams

(

array $

params

)

[protected]

Get headers to send to Swift when reading a file based on a FileBackend params array, e.g.

that of getLocalCopy(). $params is currently only checked for a 'latest' flag.

Parameters:

array

$params

Returns:

Array

Definition at line 1250 of file SwiftFileBackend.php.

SwiftFileBackend::isPathUsableInternal

(

$

storagePath

)

Check if a file can be created or changed at a given storage path.

FS backends should check if the parent directory exists, files can be written under it, and that any file already there is writable. Backends using key/value stores should check if the container exists.

Parameters:

string

$storagePath

Returns:

bool

Reimplemented from FileBackendStore.

Definition at line 170 of file SwiftFileBackend.php.

SwiftFileBackend::loadObjectListing	(	array $	params,
		$	dir,
		array $	cfObjects
	)		[private]

Load a list of objects that belong under $dir into stat cache and return a list of the names of the objects in the same order.

Parameters:

array	$params	Parameters for getDirectoryList()
string	$dir	Resolved container directory path
array	$cfObjects	List of CF_Object items

Returns:

array List of object names

Definition at line 1057 of file SwiftFileBackend.php.

SwiftFileBackend::purgeCDNCache

(

array $

objects

)

Purge the CDN cache of affected objects if CDN caching is enabled.

This is for Rackspace/Akamai CDNs.

Parameters:

array

$objects

List of CF_Object items

Returns:

void

Definition at line 1333 of file SwiftFileBackend.php.

SwiftFileBackend::resolveContainerPath	(	$	container,
		$	relStoragePath
	)		[protected]

See also:

FileBackendStore::resolveContainerPath()

Returns:

null

Reimplemented from FileBackendStore.

Definition at line 161 of file SwiftFileBackend.php.

SwiftFileBackend::sanitizeHdrs

(

array $

headers

)

[protected]

Parameters:

array

$headers

Returns:

array

Definition at line 191 of file SwiftFileBackend.php.

References truncDisp().

SwiftFileBackend::setContainerAccess	(	CF_Container $	contObj,
		array $	readGrps,
		array $	writeGrps
	)		[protected]

Set read/write permissions for a Swift container.

$readGrps is a list of the possible criteria for a request to have access to read a container. Each item is one of the following formats:

• account:user : Grants access if the request is by the given user
• ".r:<regex>" : Grants access if the request is from a referrer host that matches the expression and the request is not for a listing. Setting this to '*' effectively makes a container public. -".rlistings:<regex>" : Grants access if the request is from a referrer host that matches the expression and the request is for a listing.

$writeGrps is a list of the possible criteria for a request to have access to write to a container. Each item is of the following format:

• account:user : Grants access if the request is by the given user

See also:

http://swift.openstack.org/misc.html#acls

In general, we don't allow listings to end-users. It's not useful, isn't well-defined (lists are truncated to 10000 item with no way to page), and is just a performance risk.

Parameters:

CF_Container	$contObj	Swift container
array	$readGrps	List of read access routes
array	$writeGrps	List of write access routes

Returns:

Status

Definition at line 1310 of file SwiftFileBackend.php.

Referenced by doPublishInternal(), and doSecureInternal().

SwiftFileBackend::truncDisp

(

$

disposition

)

[protected]

Parameters:

string

$disposition

Content-Disposition header value

Returns:

string Truncated Content-Disposition header value to meet Swift limits

Definition at line 203 of file SwiftFileBackend.php.

References $res, and as.

Referenced by sanitizeHdrs().

---

Member Data Documentation

CF_Authentication SwiftFileBackend::$auth [protected]

*

Definition at line 40 of file SwiftFileBackend.php.

SwiftFileBackend::$authTTL [protected]

Definition at line 41 of file SwiftFileBackend.php.

CF_Connection SwiftFileBackend::$conn [protected]

*

Definition at line 52 of file SwiftFileBackend.php.

ProcessCacheLRU SwiftFileBackend::$connContainerCache [protected]

*

Definition at line 60 of file SwiftFileBackend.php.

SwiftFileBackend::$connErrorTime = 0 [protected]

Definition at line 56 of file SwiftFileBackend.php.

CloudFilesException SwiftFileBackend::$connException [protected]

*

Definition at line 55 of file SwiftFileBackend.php.

SwiftFileBackend::$rgwS3AccessKey [protected]

Definition at line 49 of file SwiftFileBackend.php.

SwiftFileBackend::$rgwS3SecretKey [protected]

Definition at line 50 of file SwiftFileBackend.php.

SwiftFileBackend::$sessionStarted = 0 [protected]

Definition at line 53 of file SwiftFileBackend.php.

BagOStuff SwiftFileBackend::$srvCache [protected]

*

Definition at line 58 of file SwiftFileBackend.php.

SwiftFileBackend::$swiftAnonUser [protected]

Definition at line 43 of file SwiftFileBackend.php.

SwiftFileBackend::$swiftCDNExpiry [protected]

Definition at line 45 of file SwiftFileBackend.php.

SwiftFileBackend::$swiftCDNPurgable [protected]

Definition at line 46 of file SwiftFileBackend.php.

SwiftFileBackend::$swiftTempUrlKey [protected]

Definition at line 42 of file SwiftFileBackend.php.

SwiftFileBackend::$swiftUseCDN [protected]

Definition at line 44 of file SwiftFileBackend.php.

---

The documentation for this class was generated from the following file:

• includes/filebackend/SwiftFileBackend.php