Class for an OpenStack Swift (or Ceph RGW) based file backend. More...

Inheritance diagram for SwiftFileBackend:

Collaboration diagram for SwiftFileBackend:

Public Member Functions
	__construct (array $config)
	getDirectoryListInternal ($fullCont, $dir, array $params)
	getDirListPageInternal ($fullCont, $dir, &$after, $limit, array $params)
	Do not call this function outside of SwiftFileBackendFileList.
	getFeatures ()
	Get the a bitfield of extra features supported by the backend medium.
	getFileHttpUrl (array $params)
	getFileListInternal ($fullCont, $dir, array $params)
	getFileListPageInternal ($fullCont, $dir, &$after, $limit, array $params)
	Do not call this function outside of SwiftFileBackendFileList.
	isPathUsableInternal ($storagePath)
	Check if a file can be created or changed at a given storage path.
	loadListingStatInternal ($path, array $val)
	Do not call this function outside of SwiftFileBackendFileList.
	onError ($status, $func, array $params, $err= '', $code=0, $desc= '')
	Log an unexpected exception for this backend.
Protected Member Functions
	addMissingMetadata (array $objHdrs, $path)
	Fill in any missing object metadata and save it to Swift.
	authTokenHeaders (array $creds)
	convertSwiftDate ($ts, $format=TS_MW)
	Convert dates like "Tue, 03 Jan 2012 22:01:04 GMT"/"2013-05-11T07:37:27.678360Z".
	createContainer ($container, array $params)
	Create a Swift container.
	deleteContainer ($container, array $params)
	Delete a Swift container.
	directoriesAreVirtual ()
	Is this a key/value store where directories are just virtual? Virtual directories exists in so much as files exists that are prefixed with the directory path followed by a forward slash.
	doCleanInternal ($fullCont, $dir, array $params)
	doCopyInternal (array $params)
	doCreateInternal (array $params)
	doDeleteInternal (array $params)
	doDescribeInternal (array $params)
	doDirectoryExists ($fullCont, $dir, array $params)
	doExecuteOpHandlesInternal (array $fileOpHandles)
	doGetFileContentsMulti (array $params)
	doGetFileSha1base36 (array $params)
	doGetFileStat (array $params)
	doGetFileStatMulti (array $params)
	Get file stat information (concurrently if possible) for several files.
	doGetFileXAttributes (array $params)
	doGetLocalCopyMulti (array $params)
	doMoveInternal (array $params)
	doPrepareInternal ($fullCont, $dir, array $params)
	doPrimeContainerCache (array $containerInfo)
	Fill the backend-specific process cache given an array of resolved container names and their corresponding cached info.
	doPublishInternal ($fullCont, $dir, array $params)
	doSecureInternal ($fullCont, $dir, array $params)
	doStoreInternal (array $params)
	doStreamFile (array $params)
	getAuthentication ()
	getContainerStat ($container, $bypassCache=false)
	Get a Swift container stat array, possibly from process cache.
	headersFromParams (array $params)
	Get headers to send to Swift when reading a file based on a FileBackend params array, e.g.
	resolveContainerPath ($container, $relStoragePath)
	Resolve a relative storage path, checking if it's allowed by the backend.
	sanitizeHdrs (array $params)
	Sanitize and filter the custom headers from a $params array.
	setContainerAccess ($container, array $readGrps, array $writeGrps)
	Set read/write permissions for a Swift container.
	storageUrl (array $creds, $container=null, $object=null)
Protected Attributes
array	$authCreds
	*
int	$authErrorTimestamp = null
	UNIX timestamp *.
int	$authSessionTimestamp = 0
	UNIX timestamp *.
int	$authTTL
	TTL in seconds *.
ProcessCacheLRU	$containerStatCache
	Container stat cache *.
MultiHttpClient	$http
	*
bool	$isRGW = false
	Whether the server is an Ceph RGW *.
string	$rgwS3AccessKey
	S3 access key (RADOS Gateway) *.
string	$rgwS3SecretKey
	S3 authentication key (RADOS Gateway) *.
BagOStuff	$srvCache
	*
string	$swiftAuthUrl
	Authentication base URL (without version) *.
string	$swiftKey
	Secret key for user *.
string	$swiftTempUrlKey
	Shared secret value for making temp URLs *.
string	$swiftUser
	Swift user (account:user) to authenticate as *.
Private Member Functions
	buildFileObjectListing (array $params, $dir, array $objects)
	Build a list of file objects, filtering out any directories and extracting any stat info if provided in $objects (for CF_Objects)
	getCredsCacheKey ($username)
	Get the cache key for a container.
	objectListing ($fullCont, $type, $limit, $after=null, $prefix=null, $delim=null)
	Get a list of objects under a container.

Detailed Description

Class for an OpenStack Swift (or Ceph RGW) based file backend.

Status messages should avoid mentioning the Swift account name. Likewise, error suppression should be used to avoid path disclosure.

Since:: 1.19

Definition at line 35 of file SwiftFileBackend.php.

Constructor & Destructor Documentation

SwiftFileBackend::__construct ( array $ config )

See also:

FileBackendStore::__construct() Additional $config params include:

swiftAuthUrl : Swift authentication server URL
swiftUser : Swift user used by MediaWiki (account:username)
swiftKey : Swift authentication key for the above user
swiftAuthTTL : Swift authentication TTL (seconds)
swiftTempUrlKey : Swift "X-Account-Meta-Temp-URL-Key" value on the account. Do not set this until it has been set in the backend.
shardViaHashLevels : Map of container names to sharding config with:
- base : base of hash characters, 16 or 36
- levels : the number of hash levels (and digits)
- repeat : hash subdirectories are prefixed with all the parent hash directory names (e.g. "a/ab/abc")
cacheAuthInfo : Whether to cache authentication tokens in APC, XCache, ect. If those are not available, then the main cache will be used. This is probably insecure in shared hosting environments.
rgwS3AccessKey : Rados Gateway S3 "access key" value on the account. Do not set this until it has been set in the backend. This is used for generating expiring pre-authenticated URLs. Only use this when using rgw and to work around http://tracker.newdream.net/issues/3454.
rgwS3SecretKey : Rados Gateway S3 "secret key" value on the account. Do not set this until it has been set in the backend. This is used for generating expiring pre-authenticated URLs. Only use this when using rgw and to work around http://tracker.newdream.net/issues/3454.

Reimplemented from FileBackendStore.

Definition at line 92 of file SwiftFileBackend.php.

Member Function Documentation

SwiftFileBackend::addMissingMetadata	(	array $	objHdrs,
		$	path
	)		`[protected]`

Fill in any missing object metadata and save it to Swift.

Parameters:

array	$objHdrs	Object response headers
string	$path	Storage path to object

Returns:: array New headers

Definition at line 644 of file SwiftFileBackend.php.

SwiftFileBackend::authTokenHeaders ( array $ creds ) [protected]

Parameters:

array $creds From getAuthentication()

Returns:: array

Definition at line 1613 of file SwiftFileBackend.php.

SwiftFileBackend::buildFileObjectListing	(	array $	params,
		$	dir,
		array $	objects
	)		`[private]`

Build a list of file objects, filtering out any directories and extracting any stat info if provided in $objects (for CF_Objects)

Parameters:

array	$params	Parameters for getDirectoryList()
string	$dir	Resolved container directory path
array	$objects	List of CF_Object items or object names

Returns:: array List of (names,stat array or null) entries

Definition at line 911 of file SwiftFileBackend.php.

SwiftFileBackend::convertSwiftDate	(	$	ts,
		$	format = `TS_MW`
	)		`[protected]`

Convert dates like "Tue, 03 Jan 2012 22:01:04 GMT"/"2013-05-11T07:37:27.678360Z".

Dates might also come in like "2013-05-11T07:37:27.678360" from Swift listings, missing the timezone suffix (though Ceph RGW does not appear to have this bug).

Parameters:

string	$ts
int	$format	Output format (TS_* constant)

Returns:: string

Exceptions:

FileBackendError

Definition at line 627 of file SwiftFileBackend.php.

SwiftFileBackend::createContainer	(	$	container,
		array $	params
	)		`[protected]`

Create a Swift container.

Parameters:

string	$container	Container name
array	$params

Returns:: Status

Definition at line 1308 of file SwiftFileBackend.php.

SwiftFileBackend::deleteContainer	(	$	container,
		array $	params
	)		`[protected]`

Delete a Swift container.

Parameters:

string	$container	Container name
array	$params

Returns:: Status

Definition at line 1353 of file SwiftFileBackend.php.

SwiftFileBackend::directoriesAreVirtual ( ) [protected]

Is this a key/value store where directories are just virtual? Virtual directories exists in so much as files exists that are prefixed with the directory path followed by a forward slash.

Returns:: bool

Reimplemented from FileBackendStore.

Definition at line 1129 of file SwiftFileBackend.php.

SwiftFileBackend::doCleanInternal	(	$	container,
		$	dir,
		array $	params
	)		`[protected]`

See also:: FileBackendStore::doClean()

Parameters:

string	$container
string	$dir
array	$params

Returns:: Status

Reimplemented from FileBackendStore.

Definition at line 582 of file SwiftFileBackend.php.

SwiftFileBackend::doCopyInternal ( array $ params ) [protected]

See also:: FileBackendStore::copyInternal()

Parameters:

array $params

Returns:: Status

Reimplemented from FileBackendStore.

Definition at line 310 of file SwiftFileBackend.php.

SwiftFileBackend::doCreateInternal ( array $ params ) [protected]

See also:: FileBackendStore::createInternal()

Parameters:

array $params

Returns:: Status

Reimplemented from FileBackendStore.

Definition at line 199 of file SwiftFileBackend.php.

SwiftFileBackend::doDeleteInternal ( array $ params ) [protected]

See also:: FileBackendStore::deleteInternal()

Parameters:

array $params

Returns:: Status

Reimplemented from FileBackendStore.

Definition at line 419 of file SwiftFileBackend.php.

SwiftFileBackend::doDescribeInternal ( array $ params ) [protected]

See also:: FileBackendStore::describeInternal()

Parameters:

array $params

Returns:: Status

Reimplemented from FileBackendStore.

Definition at line 460 of file SwiftFileBackend.php.

SwiftFileBackend::doDirectoryExists	(	$	container,
		$	dir,
		array $	params
	)		`[protected]`

See also:: FileBackendStore::directoryExists()

Parameters:

string	$container	Resolved container name
string	$dir	Resolved path relative to container
array	$params

Returns:: bool|null

Reimplemented from FileBackendStore.

Definition at line 734 of file SwiftFileBackend.php.

SwiftFileBackend::doExecuteOpHandlesInternal ( array $ fileOpHandles ) [protected]

See also:: FileBackendStore::executeOpHandlesInternal()

Parameters:

array $fileOpHandles

Exceptions:

FileBackendError

Returns:: array List of corresponding Status objects

Reimplemented from FileBackendStore.

Definition at line 1150 of file SwiftFileBackend.php.

SwiftFileBackend::doGetFileContentsMulti ( array $ params ) [protected]

See also:: FileBackendStore::getFileContentsMulti()

Parameters:

array $params

Returns:: array

Reimplemented from FileBackendStore.

Definition at line 685 of file SwiftFileBackend.php.

SwiftFileBackend::doGetFileSha1base36 ( array $ params ) [protected]

Definition at line 961 of file SwiftFileBackend.php.

SwiftFileBackend::doGetFileStat ( array $ params ) [protected]

See also:: FileBackendStore::getFileStat()

Reimplemented from FileBackendStore.

Definition at line 609 of file SwiftFileBackend.php.

SwiftFileBackend::doGetFileStatMulti ( array $ params ) [protected]

Get file stat information (concurrently if possible) for several files.

See also:: FileBackend::getFileStat()

Parameters:

array

$params

Parameters include:

srcs : list of source storage paths
latest : use the latest available data

Returns:: array|null Map of storage paths to array|bool|null (returns null if not supported)

Since:: 1.23

Reimplemented from FileBackendStore.

Definition at line 1451 of file SwiftFileBackend.php.

SwiftFileBackend::doGetFileXAttributes ( array $ params ) [protected]

See also:: FileBackendStore::getFileXAttributes()

Returns:: bool|string

Reimplemented from FileBackendStore.

Definition at line 946 of file SwiftFileBackend.php.

SwiftFileBackend::doGetLocalCopyMulti ( array $ params ) [protected]

See also:: FileBackendStore::getLocalCopyMulti()

Parameters:

array $params

Returns:: array

Reimplemented from FileBackendStore.

Definition at line 1012 of file SwiftFileBackend.php.

SwiftFileBackend::doMoveInternal ( array $ params ) [protected]

See also:: FileBackendStore::moveInternal()

Parameters:

array $params

Returns:: Status

Reimplemented from FileBackendStore.

Definition at line 359 of file SwiftFileBackend.php.

SwiftFileBackend::doPrepareInternal	(	$	container,
		$	dir,
		array $	params
	)		`[protected]`

See also:: FileBackendStore::doPrepare()

Parameters:

string	$container
string	$dir
array	$params

Returns:: Status

Reimplemented from FileBackendStore.

Definition at line 517 of file SwiftFileBackend.php.

SwiftFileBackend::doPrimeContainerCache ( array $ containerInfo ) [protected]

Fill the backend-specific process cache given an array of resolved container names and their corresponding cached info.

Only containers that actually exist should appear in the map.

Parameters:

array $containerInfo Map of resolved container names to cached info

Reimplemented from FileBackendStore.

Definition at line 1445 of file SwiftFileBackend.php.

SwiftFileBackend::doPublishInternal	(	$	container,
		$	dir,
		array $	params
	)		`[protected]`

See also:: FileBackendStore::doPublish()

Parameters:

string	$container
string	$dir
array	$params

Returns:: Status

Reimplemented from FileBackendStore.

Definition at line 562 of file SwiftFileBackend.php.

SwiftFileBackend::doSecureInternal	(	$	container,
		$	dir,
		array $	params
	)		`[protected]`

See also:: FileBackendStore::doSecure()

Parameters:

string	$container
string	$dir
array	$params

Returns:: Status

Reimplemented from FileBackendStore.

Definition at line 539 of file SwiftFileBackend.php.

SwiftFileBackend::doStoreInternal ( array $ params ) [protected]

See also:: FileBackendStore::storeInternal()

Parameters:

array $params

Returns:: Status

Reimplemented from FileBackendStore.

Definition at line 247 of file SwiftFileBackend.php.

SwiftFileBackend::doStreamFile ( array $ params ) [protected]

See also:: FileBackendStore::streamFile()

Parameters:

array $params

Returns:: Status

Reimplemented from FileBackendStore.

Definition at line 976 of file SwiftFileBackend.php.

SwiftFileBackend::getAuthentication ( ) [protected]

Returns:: array|null Credential map

Definition at line 1533 of file SwiftFileBackend.php.

SwiftFileBackend::getContainerStat	(	$	container,
		$	bypassCache = `false`
	)		`[protected]`

Get a Swift container stat array, possibly from process cache.

Use $reCache if the file count or byte count is needed.

Parameters:

string	$container	Container name
bool	$bypassCache	Bypass all caches and load from Swift

Returns:: array|bool|null False on 404, null on failure

Definition at line 1255 of file SwiftFileBackend.php.

SwiftFileBackend::getCredsCacheKey ( $ username ) [private]

Get the cache key for a container.

Parameters:

string $username

Returns:: string

Definition at line 1623 of file SwiftFileBackend.php.

SwiftFileBackend::getDirectoryListInternal	(	$	fullCont,
		$	dir,
		array $	params
	)

See also:: FileBackendStore::getDirectoryListInternal()

Parameters:

string	$fullCont
string	$dir
array	$params

Returns:: SwiftFileBackendDirList

Reimplemented from FileBackendStore.

Definition at line 751 of file SwiftFileBackend.php.

SwiftFileBackend::getDirListPageInternal	(	$	fullCont,
		$	dir,
		&$	after,
		$	limit,
		array $	params
	)

Do not call this function outside of SwiftFileBackendFileList.

Parameters:

string	$fullCont	Resolved container name
string	$dir	Resolved storage directory with no trailing slash
string \| null	$after	Resolved container relative path to list items after
int	$limit	Max number of items to list
array	$params	Parameters for getDirectoryList()

Returns:: array List of container relative resolved paths of directories directly under $dir

Exceptions:

FileBackendError

Definition at line 777 of file SwiftFileBackend.php.

SwiftFileBackend::getFeatures ( )

Get the a bitfield of extra features supported by the backend medium.

Returns:: int Bitfield of FileBackend::ATTR_* flags

Since:: 1.23

Reimplemented from FileBackend.

Definition at line 134 of file SwiftFileBackend.php.

References wfGetMainCache().

SwiftFileBackend::getFileHttpUrl ( array $ params )

See also:: FileBackend::getFileHttpUrl()

Parameters:

array $params

Returns:: string|null

Reimplemented from FileBackendStore.

Definition at line 1075 of file SwiftFileBackend.php.

SwiftFileBackend::getFileListInternal	(	$	fullCont,
		$	dir,
		array $	params
	)

See also:: FileBackendStore::getFileListInternal()

Parameters:

string	$fullCont
string	$dir
array	$params

Returns:: SwiftFileBackendFileList

Reimplemented from FileBackendStore.

Definition at line 762 of file SwiftFileBackend.php.

SwiftFileBackend::getFileListPageInternal	(	$	fullCont,
		$	dir,
		&$	after,
		$	limit,
		array $	params
	)

Do not call this function outside of SwiftFileBackendFileList.

Parameters:

string	$fullCont	Resolved container name
string	$dir	Resolved storage directory with no trailing slash
string \| null	$after	Resolved container relative path of file to list items after
int	$limit	Max number of items to list
array	$params	Parameters for getDirectoryList()

Returns:: array List of resolved container relative paths of files under $dir

Exceptions:

FileBackendError

Definition at line 857 of file SwiftFileBackend.php.

SwiftFileBackend::headersFromParams ( array $ params ) [protected]

Get headers to send to Swift when reading a file based on a FileBackend params array, e.g.

that of getLocalCopy(). $params is currently only checked for a 'latest' flag.

Parameters:

array $params

Returns:: array

Definition at line 1141 of file SwiftFileBackend.php.

SwiftFileBackend::isPathUsableInternal ( $ storagePath )

Check if a file can be created or changed at a given storage path.

FS backends should check if the parent directory exists, files can be written under it, and that any file already there is writable. Backends using key/value stores should check if the container exists.

Parameters:

string $storagePath

Returns:: bool

Reimplemented from FileBackendStore.

Definition at line 149 of file SwiftFileBackend.php.

SwiftFileBackend::loadListingStatInternal	(	$	path,
		array $	val
	)

Do not call this function outside of SwiftFileBackendFileList.

Parameters:

string	$path	Storage path
array	$val	Stat value

Definition at line 942 of file SwiftFileBackend.php.

SwiftFileBackend::objectListing	(	$	fullCont,
		$	type,
		$	limit,
		$	after = `null`,
		$	prefix = `null`,
		$	delim = `null`
	)		`[private]`

Get a list of objects under a container.

Either just the names or a list of stdClass objects with details can be returned.

Parameters:

string	$fullCont
string	$type	('info' for a list of object detail maps, 'names' for names only)
int	$limit
string \| null	$after
string \| null	$prefix
string \| null	$delim

Returns:: Status With the list as value

Definition at line 1394 of file SwiftFileBackend.php.

SwiftFileBackend::onError	(	$	status,
		$	func,
		array $	params,
		$	err = `''`,
		$	code = `0`,
		$	desc = `''`
	)

Log an unexpected exception for this backend.

This also sets the Status object to have a fatal error.

Parameters:

Status \| null	$status
string	$func
array	$params
string	$err	Error string
int	$code	HTTP status
string	$desc	HTTP status description

Definition at line 1638 of file SwiftFileBackend.php.

SwiftFileBackend::resolveContainerPath	(	$	container,
		$	relStoragePath
	)		`[protected]`

Resolve a relative storage path, checking if it's allowed by the backend.

This is intended for internal use, such as encoding illegal chars or perhaps getting absolute paths (e.g. FS based backends). Note that the relative path may be the empty string (e.g. the path is simply to the container).

Parameters:

string	$container	Container name
string	$relStoragePath	Storage path relative to the container

Returns:: string|null Path or null if not valid

Reimplemented from FileBackendStore.

Definition at line 139 of file SwiftFileBackend.php.

References array(), and ObjectCache\newAccelerator().

SwiftFileBackend::sanitizeHdrs ( array $ params ) [protected]

Sanitize and filter the custom headers from a $params array.

We only allow certain Content- and X-Content- headers.

Parameters:

array $params

Returns:: array Sanitized value of 'headers' field in $params

Definition at line 165 of file SwiftFileBackend.php.

SwiftFileBackend::setContainerAccess	(	$	container,
		array $	readGrps,
		array $	writeGrps
	)		`[protected]`

Set read/write permissions for a Swift container.

See also:: http://swift.openstack.org/misc.html#acls

In general, we don't allow listings to end-users. It's not useful, isn't well-defined (lists are truncated to 10000 item with no way to page), and is just a performance risk.

Parameters:

string	$container	Resolved Swift container
array	$readGrps	List of the possible criteria for a request to have access to read a container. Each item is one of the following formats: account:user : Grants access if the request is by the given user ".r:<regex>" : Grants access if the request is from a referrer host that matches the expression and the request is not for a listing. Setting this to '*' effectively makes a container public. -".rlistings:<regex>" : Grants access if the request is from a referrer host that matches the expression and the request is for a listing.
array	$writeGrps	A list of the possible criteria for a request to have access to write to a container. Each item is of the following format: account:user : Grants access if the request is by the given user

Returns:: Status

Definition at line 1221 of file SwiftFileBackend.php.

SwiftFileBackend::storageUrl	(	array $	creds,
		$	container = `null`,
		$	object = `null`
	)		`[protected]`

Parameters:

array	$creds	From getAuthentication()
string	$container
string	$object

Returns:: array

Definition at line 1597 of file SwiftFileBackend.php.

Member Data Documentation

array SwiftFileBackend::$authCreds [protected]

*

Definition at line 56 of file SwiftFileBackend.php.

int SwiftFileBackend::$authErrorTimestamp = null [protected]

UNIX timestamp *.

Definition at line 60 of file SwiftFileBackend.php.

int SwiftFileBackend::$authSessionTimestamp = 0 [protected]

UNIX timestamp *.

Definition at line 58 of file SwiftFileBackend.php.

int SwiftFileBackend::$authTTL [protected]

TTL in seconds *.

Definition at line 38 of file SwiftFileBackend.php.

ProcessCacheLRU SwiftFileBackend::$containerStatCache [protected]

Container stat cache *.

Definition at line 54 of file SwiftFileBackend.php.

MultiHttpClient SwiftFileBackend::$http [protected]

*

Definition at line 36 of file SwiftFileBackend.php.

bool SwiftFileBackend::$isRGW = false [protected]

Whether the server is an Ceph RGW *.

Definition at line 62 of file SwiftFileBackend.php.

string SwiftFileBackend::$rgwS3AccessKey [protected]

S3 access key (RADOS Gateway) *.

Definition at line 48 of file SwiftFileBackend.php.

string SwiftFileBackend::$rgwS3SecretKey [protected]

S3 authentication key (RADOS Gateway) *.

Definition at line 50 of file SwiftFileBackend.php.

BagOStuff SwiftFileBackend::$srvCache [protected]

*

Definition at line 52 of file SwiftFileBackend.php.

string SwiftFileBackend::$swiftAuthUrl [protected]

Authentication base URL (without version) *.

Definition at line 40 of file SwiftFileBackend.php.

string SwiftFileBackend::$swiftKey [protected]

Secret key for user *.

Definition at line 44 of file SwiftFileBackend.php.

string SwiftFileBackend::$swiftTempUrlKey [protected]

Shared secret value for making temp URLs *.

Definition at line 46 of file SwiftFileBackend.php.

string SwiftFileBackend::$swiftUser [protected]

Swift user (account:user) to authenticate as *.

Definition at line 42 of file SwiftFileBackend.php.

The documentation for this class was generated from the following file:

includes/filebackend/SwiftFileBackend.php

Public Member Functions

Protected Member Functions

Protected Attributes

Private Member Functions

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation