MediaWiki  REL1_21
SwiftFileBackend Class Reference

Class for an OpenStack Swift based file backend. More...

Inheritance diagram for SwiftFileBackend:
Collaboration diagram for SwiftFileBackend:

List of all members.

Public Member Functions

 __construct (array $config)
 getDirectoryListInternal ($fullCont, $dir, array $params)
 getDirListPageInternal ($fullCont, $dir, &$after, $limit, array $params)
 Do not call this function outside of SwiftFileBackendFileList.
 getFileHttpUrl (array $params)
 getFileListInternal ($fullCont, $dir, array $params)
 getFileListPageInternal ($fullCont, $dir, &$after, $limit, array $params)
 Do not call this function outside of SwiftFileBackendFileList.
 isPathUsableInternal ($storagePath)
 purgeCDNCache (array $objects)
 Purge the CDN cache of affected objects if CDN caching is enabled.

Protected Member Functions

 _getResponseCopy (CF_Async_Op $cfOp, Status $status, array $params)
 _getResponseCreate (CF_Async_Op $cfOp, Status $status, array $params)
 _getResponseDelete (CF_Async_Op $cfOp, Status $status, array $params)
 _getResponseMove (CF_Async_Op $cfOp, Status $status, array $params)
 _getResponseStore (CF_Async_Op $cfOp, Status $status, array $params)
 addMissingMetadata (CF_Object $obj, $path)
 Fill in any missing object metadata and save it to Swift.
 closeConnection ()
 Close the connection to the Swift proxy.
 createContainer ($container)
 Create a Swift container.
 deleteContainer ($container)
 Delete a Swift container.
 directoriesAreVirtual ()
 doCleanInternal ($fullCont, $dir, array $params)
 doCopyInternal (array $params)
 doCreateInternal (array $params)
 doDeleteInternal (array $params)
 doDescribeInternal (array $params)
 doDirectoryExists ($fullCont, $dir, array $params)
 doExecuteOpHandlesInternal (array $fileOpHandles)
 doGetFileContentsMulti (array $params)
 doGetFileSha1base36 (array $params)
 doGetFileStat (array $params)
 doGetLocalCopyMulti (array $params)
 doMoveInternal (array $params)
 doPrepareInternal ($fullCont, $dir, array $params)
 doPrimeContainerCache (array $containerInfo)
 doPublishInternal ($fullCont, $dir, array $params)
 doSecureInternal ($fullCont, $dir, array $params)
 doStoreInternal (array $params)
 doStreamFile (array $params)
 getConnection ()
 Get an authenticated connection handle to the Swift proxy.
 getContainer ($container, $bypassCache=false)
 Get a Swift container object, possibly from process cache.
 getParentDir ($path)
 handleException (Exception $e, $status, $func, array $params)
 Log an unexpected exception for this backend.
 headersFromParams (array $params)
 Get headers to send to Swift when reading a file based on a FileBackend params array, e.g.
 resolveContainerPath ($container, $relStoragePath)
 setContainerAccess (CF_Container $contObj, array $readGrps, array $writeGrps)
 Set read/write permissions for a Swift container.
 truncDisp ($disposition)

Protected Attributes

CF_Authentication $auth
 *
 $authTTL
CF_Connection $conn
 *
ProcessCacheLRU $connContainerCache
 *
 $connErrorTime = 0
CloudFilesException $connException
 *
 $rgwS3AccessKey
 $rgwS3SecretKey
 $sessionStarted = 0
BagOStuff $srvCache
 *
 $swiftAnonUser
 $swiftCDNExpiry
 $swiftCDNPurgable
 $swiftTempUrlKey
 $swiftUseCDN

Private Member Functions

 getCredsCacheKey ($username)
 Get the cache key for a container.

Detailed Description

Class for an OpenStack Swift based file backend.

This requires the SwiftCloudFiles MediaWiki extension, which includes the php-cloudfiles library (https://github.com/rackspace/php-cloudfiles). php-cloudfiles requires the curl, fileinfo, and mb_string PHP extensions.

Status messages should avoid mentioning the Swift account name. Likewise, error suppression should be used to avoid path disclosure.

Since:
1.19

Definition at line 39 of file SwiftFileBackend.php.


Constructor & Destructor Documentation

See also:
FileBackendStore::__construct() Additional $config params include:
  • swiftAuthUrl : Swift authentication server URL
  • swiftUser : Swift user used by MediaWiki (account:username)
  • swiftKey : Swift authentication key for the above user
  • swiftAuthTTL : Swift authentication TTL (seconds)
  • swiftTempUrlKey : Swift "X-Account-Meta-Temp-URL-Key" value on the account. Do not set this until it has been set in the backend.
  • swiftAnonUser : Swift user used for end-user requests (account:username). If set, then views of public containers are assumed to go through this user. If not set, then public containers are accessible to unauthenticated requests via ".r:*" in the ACL.
  • swiftUseCDN : Whether a Cloud Files Content Delivery Network is set up
  • swiftCDNExpiry : How long (in seconds) to store content in the CDN. If files may likely change, this should probably not exceed a few days. For example, deletions may take this long to apply. If object purging is enabled, however, this is not an issue.
  • swiftCDNPurgable : Whether object purge requests are allowed by the CDN.
  • shardViaHashLevels : Map of container names to sharding config with:
  • cacheAuthInfo : Whether to cache authentication tokens in APC, XCache, ect. If those are not available, then the main cache will be used. This is probably insecure in shared hosting environments.
  • rgwS3AccessKey : Ragos Gateway S3 "access key" value on the account. Do not set this until it has been set in the backend. This is used for generating expiring pre-authenticated URLs. Only use this when using rgw and to work around http://tracker.newdream.net/issues/3454.
  • rgwS3SecretKey : Ragos Gateway S3 "secret key" value on the account. Do not set this until it has been set in the backend. This is used for generating expiring pre-authenticated URLs. Only use this when using rgw and to work around http://tracker.newdream.net/issues/3454.

Reimplemented from FileBackendStore.

Definition at line 100 of file SwiftFileBackend.php.


Member Function Documentation

SwiftFileBackend::_getResponseCopy ( CF_Async_Op $  cfOp,
Status status,
array params 
) [protected]
SwiftFileBackend::_getResponseCreate ( CF_Async_Op $  cfOp,
Status status,
array params 
) [protected]
SwiftFileBackend::_getResponseDelete ( CF_Async_Op $  cfOp,
Status status,
array params 
) [protected]
SwiftFileBackend::_getResponseMove ( CF_Async_Op $  cfOp,
Status status,
array params 
) [protected]
SwiftFileBackend::_getResponseStore ( CF_Async_Op $  cfOp,
Status status,
array params 
) [protected]
SwiftFileBackend::addMissingMetadata ( CF_Object $  obj,
path 
) [protected]

Fill in any missing object metadata and save it to Swift.

Parameters:
$objCF_Object
string$pathStorage path to object
Returns:
bool Success
Exceptions:
Exceptioncloudfiles exceptions

Definition at line 824 of file SwiftFileBackend.php.

Referenced by doGetFileStat().

Close the connection to the Swift proxy.

Returns:
void

Definition at line 1419 of file SwiftFileBackend.php.

SwiftFileBackend::createContainer ( container) [protected]

Create a Swift container.

Parameters:
string$containerContainer name
Returns:
CF_Container
Exceptions:
CloudFilesException

Definition at line 1474 of file SwiftFileBackend.php.

Referenced by doPrepareInternal().

SwiftFileBackend::deleteContainer ( container) [protected]

Delete a Swift container.

Parameters:
string$containerContainer name
Returns:
void
Exceptions:
CloudFilesException

Definition at line 1488 of file SwiftFileBackend.php.

Referenced by doCleanInternal().

See also:
FileBackendStore::directoriesAreVirtual()
Returns:
bool

Reimplemented from FileBackendStore.

Definition at line 1252 of file SwiftFileBackend.php.

SwiftFileBackend::doCleanInternal ( fullCont,
dir,
array params 
) [protected]
SwiftFileBackend::doDirectoryExists ( fullCont,
dir,
array params 
) [protected]
See also:
FileBackendStore::doDirectoryExists()
Returns:
bool|null

Reimplemented from FileBackendStore.

Definition at line 926 of file SwiftFileBackend.php.

References $dir, array(), getContainer(), and handleException().

SwiftFileBackend::doExecuteOpHandlesInternal ( array fileOpHandles) [protected]
See also:
FileBackendStore::doExecuteOpHandlesInternal()
Returns:
Array List of corresponding Status objects

Reimplemented from FileBackendStore.

Definition at line 1276 of file SwiftFileBackend.php.

References $batch, array(), as, handleException(), Status\newGood(), and purgeCDNCache().

SwiftFileBackend::doGetFileSha1base36 ( array params) [protected]
See also:
FileBackendStore::doGetFileSha1base36()
Returns:
bool

Definition at line 1082 of file SwiftFileBackend.php.

References FileBackendStore\getFileStat().

SwiftFileBackend::doGetFileStat ( array params) [protected]
See also:
FileBackendStore::doFileExists()
Returns:
array|bool|null

Reimplemented from FileBackendStore.

Definition at line 789 of file SwiftFileBackend.php.

References addMissingMetadata(), array(), getContainer(), handleException(), headersFromParams(), list, FileBackendStore\resolveStoragePathReal(), and wfTimestamp().

SwiftFileBackend::doPrimeContainerCache ( array containerInfo) [protected]
See also:
FileBackendStore::doPrimeContainerCache()
Returns:
void

Reimplemented from FileBackendStore.

Definition at line 1498 of file SwiftFileBackend.php.

References $conn, array(), as, getConnection(), and handleException().

SwiftFileBackend::doPublishInternal ( fullCont,
dir,
array params 
) [protected]
SwiftFileBackend::doSecureInternal ( fullCont,
dir,
array params 
) [protected]

Get an authenticated connection handle to the Swift proxy.

Exceptions:
CloudFilesException
CloudFilesException|Exception
Returns:
CF_Connection|bool False on failure

Definition at line 1373 of file SwiftFileBackend.php.

Referenced by doPrimeContainerCache().

SwiftFileBackend::getContainer ( container,
bypassCache = false 
) [protected]

Get a Swift container object, possibly from process cache.

Use $reCache if the file count or byte count is needed.

Parameters:
string$containerContainer name
bool$bypassCacheBypass all caches and load from Swift
Returns:
CF_Container
Exceptions:
CloudFilesException

Definition at line 1447 of file SwiftFileBackend.php.

Referenced by doCleanInternal(), doCopyInternal(), doCreateInternal(), doDeleteInternal(), doDescribeInternal(), doDirectoryExists(), doGetFileContentsMulti(), doGetFileStat(), doGetLocalCopyMulti(), doMoveInternal(), doPrepareInternal(), doPublishInternal(), doSecureInternal(), doStoreInternal(), doStreamFile(), getFileHttpUrl(), and isPathUsableInternal().

SwiftFileBackend::getCredsCacheKey ( username) [private]

Get the cache key for a container.

Parameters:
$usernamestring
Returns:
string

Definition at line 1434 of file SwiftFileBackend.php.

SwiftFileBackend::getDirectoryListInternal ( fullCont,
dir,
array params 
)
See also:
FileBackendStore::getDirectoryListInternal()
Returns:
SwiftFileBackendDirList

Reimplemented from FileBackendStore.

Definition at line 945 of file SwiftFileBackend.php.

References $dir.

SwiftFileBackend::getDirListPageInternal ( fullCont,
dir,
&$  after,
limit,
array params 
)

Do not call this function outside of SwiftFileBackendFileList.

Parameters:
string$fullContResolved container name
string$dirResolved storage directory with no trailing slash
string | null$afterStorage path of file to list items after
$limitinteger Max number of items to list
array$paramsIncludes flag for 'topOnly'
Returns:
Array List of relative paths of dirs directly under $dir

Definition at line 967 of file SwiftFileBackend.php.

SwiftFileBackend::getFileListInternal ( fullCont,
dir,
array params 
)
See also:
FileBackendStore::getFileListInternal()
Returns:
SwiftFileBackendFileList

Reimplemented from FileBackendStore.

Definition at line 953 of file SwiftFileBackend.php.

References $dir.

SwiftFileBackend::getFileListPageInternal ( fullCont,
dir,
&$  after,
limit,
array params 
)

Do not call this function outside of SwiftFileBackendFileList.

Parameters:
string$fullContResolved container name
string$dirResolved storage directory with no trailing slash
string | null$afterStorage path of file to list items after
$limitinteger Max number of items to list
array$paramsIncludes flag for 'topOnly'
Returns:
Array List of relative paths of files under $dir

Definition at line 1040 of file SwiftFileBackend.php.

SwiftFileBackend::getParentDir ( path) [protected]

Definition at line 1026 of file SwiftFileBackend.php.

SwiftFileBackend::handleException ( Exception $  e,
status,
func,
array params 
) [protected]

Log an unexpected exception for this backend.

This also sets the Status object to have a fatal error.

Parameters:
$eException
$statusStatus|null
$funcstring
array$params
Returns:
void

Definition at line 1521 of file SwiftFileBackend.php.

Referenced by doCleanInternal(), doCopyInternal(), doCreateInternal(), doDeleteInternal(), doDescribeInternal(), doDirectoryExists(), doExecuteOpHandlesInternal(), doGetFileContentsMulti(), doGetFileStat(), doGetLocalCopyMulti(), doMoveInternal(), doPrepareInternal(), doPrimeContainerCache(), doPublishInternal(), doSecureInternal(), doStoreInternal(), doStreamFile(), getFileHttpUrl(), and isPathUsableInternal().

SwiftFileBackend::headersFromParams ( array params) [protected]

Get headers to send to Swift when reading a file based on a FileBackend params array, e.g.

that of getLocalCopy(). $params is currently only checked for a 'latest' flag.

Parameters:
array$params
Returns:
Array

Definition at line 1264 of file SwiftFileBackend.php.

Referenced by doDescribeInternal(), doGetFileContentsMulti(), doGetFileStat(), doGetLocalCopyMulti(), and doStreamFile().

Purge the CDN cache of affected objects if CDN caching is enabled.

This is for Rackspace/Akamai CDNs.

Parameters:
array$objectsList of CF_Object items
Returns:
void

Definition at line 1351 of file SwiftFileBackend.php.

Referenced by doCopyInternal(), doCreateInternal(), doDeleteInternal(), doDescribeInternal(), doExecuteOpHandlesInternal(), doMoveInternal(), and doStoreInternal().

SwiftFileBackend::resolveContainerPath ( container,
relStoragePath 
) [protected]
See also:
FileBackendStore::resolveContainerPath()
Returns:
null

Reimplemented from FileBackendStore.

Definition at line 161 of file SwiftFileBackend.php.

SwiftFileBackend::setContainerAccess ( CF_Container $  contObj,
array readGrps,
array writeGrps 
) [protected]

Set read/write permissions for a Swift container.

$readGrps is a list of the possible criteria for a request to have access to read a container. Each item is one of the following formats:

  • account:user : Grants access if the request is by the given user
  • ".r:<regex>" : Grants access if the request is from a referrer host that matches the expression and the request is not for a listing. Setting this to '*' effectively makes a container public. -".rlistings:<regex>" : Grants access if the request is from a referrer host that matches the expression and the request for a listing.

$writeGrps is a list of the possible criteria for a request to have access to write to a container. Each item is of the following format:

  • account:user : Grants access if the request is by the given user
See also:
http://swift.openstack.org/misc.html#acls

In general, we don't allow listings to end-users. It's not useful, isn't well-defined (lists are truncated to 10000 item with no way to page), and is just a performance risk.

Parameters:
$contObjCF_Container Swift container
array$readGrpsList of read access routes
array$writeGrpsList of write access routes
Returns:
Status

Definition at line 1328 of file SwiftFileBackend.php.

Referenced by doPublishInternal(), and doSecureInternal().

SwiftFileBackend::truncDisp ( disposition) [protected]
Parameters:
string$dispositionContent-Disposition header value
Returns:
string Truncated Content-Disposition header value to meet Swift limits

Definition at line 195 of file SwiftFileBackend.php.

References $res, and as.

Referenced by doCopyInternal(), doCreateInternal(), doDescribeInternal(), doMoveInternal(), and doStoreInternal().


Member Data Documentation

CF_Authentication SwiftFileBackend::$auth [protected]

*

Definition at line 40 of file SwiftFileBackend.php.

SwiftFileBackend::$authTTL [protected]

Definition at line 41 of file SwiftFileBackend.php.

CF_Connection SwiftFileBackend::$conn [protected]

*

Definition at line 52 of file SwiftFileBackend.php.

Referenced by doPrimeContainerCache().

ProcessCacheLRU SwiftFileBackend::$connContainerCache [protected]

*

Definition at line 60 of file SwiftFileBackend.php.

SwiftFileBackend::$connErrorTime = 0 [protected]

Definition at line 56 of file SwiftFileBackend.php.

CloudFilesException SwiftFileBackend::$connException [protected]

*

Definition at line 55 of file SwiftFileBackend.php.

SwiftFileBackend::$rgwS3AccessKey [protected]

Definition at line 49 of file SwiftFileBackend.php.

SwiftFileBackend::$rgwS3SecretKey [protected]

Definition at line 50 of file SwiftFileBackend.php.

SwiftFileBackend::$sessionStarted = 0 [protected]

Definition at line 53 of file SwiftFileBackend.php.

BagOStuff SwiftFileBackend::$srvCache [protected]

*

Definition at line 58 of file SwiftFileBackend.php.

SwiftFileBackend::$swiftAnonUser [protected]

Definition at line 43 of file SwiftFileBackend.php.

SwiftFileBackend::$swiftCDNExpiry [protected]

Definition at line 45 of file SwiftFileBackend.php.

SwiftFileBackend::$swiftCDNPurgable [protected]

Definition at line 46 of file SwiftFileBackend.php.

SwiftFileBackend::$swiftTempUrlKey [protected]

Definition at line 42 of file SwiftFileBackend.php.

SwiftFileBackend::$swiftUseCDN [protected]

Definition at line 44 of file SwiftFileBackend.php.


The documentation for this class was generated from the following file: