Class to handle enqueueing and running of background jobs for federated queues. More...

Inheritance diagram for JobQueueFederated:

Collaboration diagram for JobQueueFederated:

Public Member Functions
	getAllDelayedJobs ()
	Get an iterator to traverse over all delayed jobs in this queue.
	getAllQueuedJobs ()
	Get an iterator to traverse over all available jobs in this queue.
	getCoalesceLocationInternal ()
	Do not use this function outside of JobQueue/JobQueueGroup.
	setTestingPrefix ($key)
	Namespace the queue with a key to isolate it for testing.
Public Attributes
const	CACHE_TTL_LONG = 300
const	CACHE_TTL_SHORT = 30
Protected Member Functions
	__construct (array $params)
	include:
	doAck (Job $job)
	doBatchPush (array $jobs, $flags)
	doDeduplicateRootJob (Job $job)
	doDelete ()
	doFlushCaches ()
	doGetAbandonedCount ()
	doGetAcquiredCount ()
	doGetDelayedCount ()
	doGetPeriodicTasks ()
	doGetSiblingQueueSizes (array $types)
	doGetSiblingQueuesWithJobs (array $types)
	doGetSize ()
	doIsEmpty ()
	doIsRootJobOldDuplicate (Job $job)
	doPop ()
	doWaitForBackups ()
	getCrossPartitionSum ($type, $method)
	optimalOrder ()
	Get the default queue order to use if configuration does not specify one.
	supportedOrders ()
	Get the allowed queue orders for configuration validation.
	supportsDelayedJobs ()
	Find out if delayed jobs are supported for configuration validation.
	tryJobInsertions (array $jobs, HashRing &$partitionRing, $flags)
Protected Attributes
BagOStuff	$cache
	*
Array	$partitionMap = array()
	(partition name => weight) reverse sorted by weight *
HashRing	$partitionPushRing
	*
Array	$partitionQueues = array()
	(partition name => JobQueue) reverse sorted by weight *
Private Member Functions
	getCacheKey ($property)

Detailed Description

Class to handle enqueueing and running of background jobs for federated queues.

This class allows for queues to be partitioned into smaller queues. A partition is defined by the configuration for a JobQueue instance. For example, one can set $wgJobTypeConf['refreshLinks'] to point to a JobQueueFederated instance, which itself would consist of three JobQueueRedis instances, each using their own redis server. This would allow for the jobs to be split (evenly or based on weights) accross multiple servers if a single server becomes impractical or expensive. Different JobQueue classes can be mixed.

The basic queue configuration (e.g. "order", "claimTTL") of a federated queue is inherited by the partition queues. Additional configuration defines what section each wiki is in, what partition queues each section uses (and their weight), and the JobQueue configuration for each partition. Some sections might only need a single queue partition, like the sections for groups of small wikis.

If used for performance, then $wgMainCacheType should be set to memcached/redis. Note that "fifo" cannot be used for the ordering, since the data is distributed. One can still use "timestamp" instead, as in "roughly timestamp ordered". Also, queue classes used by this should ignore down servers (with TTL) to avoid slowness.

Since:: 1.22

Definition at line 49 of file JobQueueFederated.php.

Constructor & Destructor Documentation

JobQueueFederated::__construct ( array $ params ) [protected]

include:

sectionsByWiki : A map of wiki IDs to section names. Wikis will default to using the section "default".
partitionsBySection : Map of section names to maps of (partition name => weight). A section called 'default' must be defined if not all wikis have explicitly defined sections.
configByPartition : Map of queue partition names to configuration arrays. These configuration arrays are passed to JobQueue::factory(). The options set here are overriden by those passed to this the federated queue itself (e.g. 'order' and 'claimTTL').
partitionsNoPush : List of partition names that can handle pop() but not push(). This can be used to migrate away from a certain partition.
Parameters:

array $params

Reimplemented from JobQueue.

Definition at line 73 of file JobQueueFederated.php.

Member Function Documentation

JobQueueFederated::doAck ( Job $ job ) [protected]

See also:: JobQueue::ack()

Returns:: bool

Reimplemented from JobQueue.

Definition at line 317 of file JobQueueFederated.php.

JobQueueFederated::doBatchPush	(	array $	jobs,
		$	flags
	)		`[protected]`

See also:: JobQueue::batchPush()

Returns:: bool

Reimplemented from JobQueue.

Definition at line 192 of file JobQueueFederated.php.

JobQueueFederated::doDeduplicateRootJob ( Job $ job ) [protected]

See also:: JobQueue::deduplicateRootJob()

Parameters:

$job Job

Returns:: bool

Reimplemented from JobQueue.

Definition at line 337 of file JobQueueFederated.php.

JobQueueFederated::doDelete ( ) [protected]

See also:: JobQueue::delete()

Returns:: bool Success

Reimplemented from JobQueue.

Definition at line 350 of file JobQueueFederated.php.

JobQueueFederated::doFlushCaches ( ) [protected]

See also:: JobQueue::flushCaches()

Returns:: void

Reimplemented from JobQueue.

Definition at line 380 of file JobQueueFederated.php.

JobQueueFederated::doGetAbandonedCount ( ) [protected]

See also:: JobQueue::getAbandonedCount()

Returns:: integer

Reimplemented from JobQueue.

Definition at line 162 of file JobQueueFederated.php.

References getCrossPartitionSum().

JobQueueFederated::doGetAcquiredCount ( ) [protected]

See also:: JobQueue::getAcquiredCount()

Returns:: integer

Reimplemented from JobQueue.

Definition at line 154 of file JobQueueFederated.php.

References getCrossPartitionSum().

JobQueueFederated::doGetDelayedCount ( ) [protected]

See also:: JobQueue::getDelayedCount()

Returns:: integer

Reimplemented from JobQueue.

Definition at line 158 of file JobQueueFederated.php.

References getCrossPartitionSum().

JobQueueFederated::doGetPeriodicTasks ( ) [protected]

See also:: JobQueue::getPeriodicTasks()

Returns:: Array

Reimplemented from JobQueue.

Definition at line 370 of file JobQueueFederated.php.

JobQueueFederated::doGetSiblingQueueSizes ( array $ types ) [protected]

See also:: JobQueue::getSiblingQueuesSize()

Parameters:

array $types List of queues types

Returns:: array|null (list of queue types) or null if unsupported

Reimplemented from JobQueue.

Definition at line 437 of file JobQueueFederated.php.

JobQueueFederated::doGetSiblingQueuesWithJobs ( array $ types ) [protected]

See also:: JobQueue::getSiblingQueuesWithJobs()

Parameters:

array $types List of queues types

Returns:: array|null (list of queue types) or null if unsupported

Reimplemented from JobQueue.

Definition at line 417 of file JobQueueFederated.php.

JobQueueFederated::doGetSize ( ) [protected]

See also:: JobQueue::getSize()

Returns:: integer

Reimplemented from JobQueue.

Definition at line 150 of file JobQueueFederated.php.

JobQueueFederated::doIsEmpty ( ) [protected]

See also:: JobQueue::isEmpty()

Returns:: bool

Reimplemented from JobQueue.

Definition at line 125 of file JobQueueFederated.php.

JobQueueFederated::doIsRootJobOldDuplicate ( Job $ job ) [protected]

See also:: JobQueue::isRootJobOldDuplicate()

Parameters:

Job $job

Returns:: bool

Reimplemented from JobQueue.

Definition at line 324 of file JobQueueFederated.php.

JobQueueFederated::doPop ( ) [protected]

See also:: JobQueue::pop()

Returns:: Job

Reimplemented from JobQueue.

Definition at line 283 of file JobQueueFederated.php.

JobQueueFederated::doWaitForBackups ( ) [protected]

See also:: JobQueue::waitForBackups()

Returns:: void

Reimplemented from JobQueue.

Definition at line 360 of file JobQueueFederated.php.

JobQueueFederated::getAllDelayedJobs ( )

Get an iterator to traverse over all delayed jobs in this queue.

Note: results may be stale if the queue is concurrently modified.

Returns:: Iterator

Exceptions:

JobQueueError

Since:: 1.22

Reimplemented from JobQueue.

Definition at line 404 of file JobQueueFederated.php.

JobQueueFederated::getAllQueuedJobs ( )

Get an iterator to traverse over all available jobs in this queue.

This does not include jobs that are currently acquired or delayed. Note: results may be stale if the queue is concurrently modified.

Returns:: Iterator

Exceptions:

JobQueueError

Reimplemented from JobQueue.

Definition at line 396 of file JobQueueFederated.php.

JobQueueFederated::getCacheKey ( $ property ) [private]

Returns:: string

Definition at line 465 of file JobQueueFederated.php.

References $property, list, type, wfForeignMemcKey(), and wfSplitWikiID().

JobQueueFederated::getCoalesceLocationInternal ( )

Do not use this function outside of JobQueue/JobQueueGroup.

Returns:: string

Since:: 1.22

Reimplemented from JobQueue.

Definition at line 412 of file JobQueueFederated.php.

JobQueueFederated::getCrossPartitionSum	(	$	type,
		$	method
	)		`[protected]`

Parameters:

string	$type
string	$method

Returns:: integer

Definition at line 171 of file JobQueueFederated.php.

Referenced by doGetAbandonedCount(), doGetAcquiredCount(), and doGetDelayedCount().

JobQueueFederated::optimalOrder ( ) [protected]

Get the default queue order to use if configuration does not specify one.

Returns:: string One of (random, timestamp, fifo, undefined)

Reimplemented from JobQueue.

Definition at line 117 of file JobQueueFederated.php.

JobQueueFederated::setTestingPrefix ( $ key )

Namespace the queue with a key to isolate it for testing.

Parameters:

$key string

Returns:: void

Exceptions:

MWException

Reimplemented from JobQueue.

Definition at line 456 of file JobQueueFederated.php.

JobQueueFederated::supportedOrders ( ) [protected]

Get the allowed queue orders for configuration validation.

Returns:: Array Subset of (random, timestamp, fifo, undefined)

Reimplemented from JobQueue.

Definition at line 112 of file JobQueueFederated.php.

JobQueueFederated::supportsDelayedJobs ( ) [protected]

Find out if delayed jobs are supported for configuration validation.

Returns:: boolean Whether delayed jobs are supported

Reimplemented from JobQueue.

Definition at line 121 of file JobQueueFederated.php.

JobQueueFederated::tryJobInsertions	(	array $	jobs,
		HashRing &$	partitionRing,
		$	flags
	)		`[protected]`

Parameters:

array	$jobs
HashRing	$partitionRing
integer	$flags

Returns:: array List of Job object that could not be inserted

Definition at line 213 of file JobQueueFederated.php.

Member Data Documentation

BagOStuff JobQueueFederated::$cache [protected]

*

Definition at line 53 of file JobQueueFederated.php.

Array JobQueueFederated::$partitionMap = array() [protected]

(partition name => weight) reverse sorted by weight *

Definition at line 50 of file JobQueueFederated.php.

HashRing JobQueueFederated::$partitionPushRing [protected]

*

Definition at line 52 of file JobQueueFederated.php.

Array JobQueueFederated::$partitionQueues = array() [protected]

(partition name => JobQueue) reverse sorted by weight *

Definition at line 51 of file JobQueueFederated.php.

const JobQueueFederated::CACHE_TTL_LONG = 300

Definition at line 56 of file JobQueueFederated.php.

const JobQueueFederated::CACHE_TTL_SHORT = 30

Definition at line 55 of file JobQueueFederated.php.

The documentation for this class was generated from the following file:

includes/job/JobQueueFederated.php

Public Member Functions

Public Attributes

Protected Member Functions

Protected Attributes

Private Member Functions

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation