Class to handle enqueueing and running of background jobs. More...

Inheritance diagram for JobQueue:

Public Member Functions
	ack (Job $job)
	Acknowledge that a job was completed.
	batchPush (array $jobs, $flags=0)
	Push a batch of jobs into the queue.
	deduplicateRootJob (Job $job)
	Register the "root job" of a given job into the queue for de-duplication.
	flushCaches ()
	Clear any process and persistent caches.
	getAcquiredCount ()
	Get the number of acquired jobs (these are temporarily out of the queue).
	getAllQueuedJobs ()
	Get an iterator to traverse over all of the jobs in this queue.
	getOrder ()
	getPeriodicTasks ()
	Return a map of task names to task definition maps.
	getSize ()
	Get the number of available (unacquired) jobs in the queue.
	getType ()
	getWiki ()
	isEmpty ()
	Quickly check if the queue is empty (has no available jobs).
	pop ()
	Pop a job off of the queue.
	push ($jobs, $flags=0)
	Push a single jobs into the queue.
	setTestingPrefix ($key)
	Namespace the queue with a key to isolate it for testing.
	waitForBackups ()
	Wait for any slaves or backup servers to catch up.
Static Public Member Functions
static	factory (array $params)
	Get a job queue object of the specified type.
Public Attributes
const	QoS_Atomic = 1
Protected Member Functions
	__construct (array $params)
	doAck (Job $job)
	doBatchPush (array $jobs, $flags)
	doDeduplicateRootJob (Job $job)
	doFlushCaches ()
	doGetAcquiredCount ()
	doGetPeriodicTasks ()
	doGetSize ()
	doIsEmpty ()
	doPop ()
	doWaitForBackups ()
	optimalOrder ()
	supportedOrders ()
Protected Attributes
	$claimTTL
	$maxTries
	$order
	$type
	$wiki

Detailed Description

Class to handle enqueueing and running of background jobs.

Since:: 1.21

Definition at line 31 of file JobQueue.php.

Constructor & Destructor Documentation

JobQueue::__construct ( array $ params ) [protected]

Parameters:

$params array

Reimplemented in JobQueueDB.

Definition at line 43 of file JobQueue.php.

References optimalOrder(), order, supportedOrders(), type, and wiki.

Member Function Documentation

JobQueue::ack ( Job $ job ) [final]

Acknowledge that a job was completed.

This does nothing for certain queue classes or if "claimTTL" is not set. Outside callers should use JobQueueGroup::ack() instead of this function.

Parameters:

$job Job

Returns:: bool

Exceptions:

MWException

Definition at line 284 of file JobQueue.php.

References $ok, doAck(), Job\getType(), type, wfProfileIn(), and wfProfileOut().

JobQueue::batchPush	(	array $	jobs,
		$	flags = `0`
	)		`[final]`

Push a batch of jobs into the queue.

This does not require $wgJobClasses to be set for the given job type. Outside callers should use JobQueueGroup::push() instead of this function.

Parameters:

array	$jobs	List of Jobs
	$flags	integer Bitfield (supports JobQueue::QoS_Atomic)

Returns:: bool Returns false on failure

Exceptions:

MWException

Definition at line 221 of file JobQueue.php.

References $flags, $job, $ok, as, doBatchPush(), type, wfProfileIn(), and wfProfileOut().

Referenced by push().

JobQueue::deduplicateRootJob ( Job $ job ) [final]

Register the "root job" of a given job into the queue for de-duplication.

This should only be called right *after* all the new jobs have been inserted. This is used to turn older, duplicate, job entries into no-ops. The root job information will remain in the registry until it simply falls out of cache.

This requires that $job has two special fields in the "params" array:

rootJobSignature : hash (e.g. SHA1) that identifies the task
rootJobTimestamp : TS_MW timestamp of this instance of the task

A "root job" is a conceptual job that consist of potentially many smaller jobs that are actually inserted into the queue. For example, "refreshLinks" jobs are spawned when a template is edited. One can think of the task as "update links of pages that use template X" and an instance of that task as a "root job". However, what actually goes into the queue are potentially many refreshLinks2 jobs. Since these jobs include things like page ID ranges and DB master positions, and morph into smaller refreshLinks2 jobs recursively, simple duplicate detection (like job_sha1) for individual jobs being identical is not useful.

In the case of "refreshLinks", if these jobs are still in the queue when the template is edited again, we want all of these old refreshLinks jobs for that template to become no-ops. This can greatly reduce server load, since refreshLinks jobs involves parsing. Essentially, the new batch of jobs belong to a new "root job" and the older ones to a previous "root job" for the same task of "update links of pages that use template X".

This does nothing for certain queue classes.

Parameters:

$job Job

Returns:: bool

Exceptions:

MWException

Definition at line 331 of file JobQueue.php.

References $ok, doDeduplicateRootJob(), Job\getType(), type, wfProfileIn(), and wfProfileOut().

JobQueue::doAck ( Job $ job ) [abstract, protected]

See also:: JobQueue::ack()

Returns:: bool

Reimplemented in JobQueueDB.

Referenced by ack().

JobQueue::doBatchPush	(	array $	jobs,
		$	flags
	)		`[abstract, protected]`

See also:: JobQueue::batchPush()

Returns:: bool

Reimplemented in JobQueueDB.

Referenced by batchPush().

JobQueue::doDeduplicateRootJob ( Job $ job ) [protected]

See also:: JobQueue::deduplicateRootJob()

Parameters:

$job Job

Returns:: bool

Reimplemented in JobQueueDB.

Definition at line 346 of file JobQueue.php.

Referenced by deduplicateRootJob().

JobQueue::doFlushCaches ( ) [protected]

See also:: JobQueue::flushCaches()

Returns:: void

Reimplemented in JobQueueDB.

Definition at line 413 of file JobQueue.php.

Referenced by flushCaches().

JobQueue::doGetAcquiredCount ( ) [abstract, protected]

See also:: JobQueue::getAcquiredCount()

Returns:: integer

Reimplemented in JobQueueDB.

Referenced by getAcquiredCount().

JobQueue::doGetPeriodicTasks ( ) [protected]

See also:: JobQueue::getPeriodicTasks()

Returns:: Array

Reimplemented in JobQueueDB.

Definition at line 394 of file JobQueue.php.

References array().

Referenced by getPeriodicTasks().

JobQueue::doGetSize ( ) [abstract, protected]

See also:: JobQueue::getSize()

Returns:: integer

Reimplemented in JobQueueDB.

Referenced by getSize().

JobQueue::doIsEmpty ( ) [abstract, protected]

See also:: JobQueue::isEmpty()

Returns:: bool

Reimplemented in JobQueueDB.

Referenced by isEmpty().

JobQueue::doPop ( ) [abstract, protected]

See also:: JobQueue::pop()

Returns:: Job

Reimplemented in JobQueueDB.

Referenced by pop().

JobQueue::doWaitForBackups ( ) [protected]

See also:: JobQueue::waitForBackups()

Returns:: void

Reimplemented in JobQueueDB.

Definition at line 368 of file JobQueue.php.

Referenced by waitForBackups().

static JobQueue::factory ( array $ params ) [static, final]

Get a job queue object of the specified type.

$params includes:

class : What job class to use (determines job type)
wiki : wiki ID of the wiki the jobs are for (defaults to current wiki)
type : The name of the job types this queue handles
order : Order that pop() selects jobs, one of "fifo", "timestamp" or "random". If "fifo" is used, the queue will effectively be FIFO. Note that job completion will not appear to be exactly FIFO if there are multiple job runners since jobs can take different times to finish once popped. If "timestamp" is used, the queue will at least be loosely ordered by timestamp, allowing for some jobs to be popped off out of order. If "random" is used, pop() will pick jobs in random order. Note that it may only be weakly random (e.g. a lottery of the oldest X). If "any" is choosen, the queue will use whatever order is the fastest. This might be useful for improving concurrency for job acquisition.
claimTTL : If supported, the queue will recycle jobs that have been popped but not acknowledged as completed after this many seconds. Recycling of jobs simple means re-inserting them into the queue. Jobs can be attempted up to three times before being discarded.

Queue classes should throw an exception if they do not support the options given.

Parameters:

$params array

Returns:: JobQueue

Exceptions:

MWException

Definition at line 85 of file JobQueue.php.

References MWInit\classExists().

Referenced by JobQueueDB\doPop(), JobQueueDB\getAllQueuedJobs(), and JobQueueTest\setUp().

JobQueue::flushCaches ( ) [final]

Clear any process and persistent caches.

Returns:: void

Definition at line 403 of file JobQueue.php.

References doFlushCaches(), wfProfileIn(), and wfProfileOut().

JobQueue::getAcquiredCount ( ) [final]

Get the number of acquired jobs (these are temporarily out of the queue).

Queue classes should use caching if they are any slower without memcached.

If caching is used, this number might be out of date for a minute.

Returns:: integer

Exceptions:

MWException

Definition at line 184 of file JobQueue.php.

References $res, doGetAcquiredCount(), wfProfileIn(), and wfProfileOut().

JobQueue::getAllQueuedJobs ( ) [abstract]

Get an iterator to traverse over all of the jobs in this queue.

This does not include jobs that are current acquired. In general, this should only be called on a queue that is no longer being popped.

Returns:: Iterator|Traversable|Array

Exceptions:

MWException

Reimplemented in JobQueueDB.

JobQueue::getOrder ( ) [final]

Returns:: string One of (random, timestamp, fifo)

Definition at line 114 of file JobQueue.php.

References order.

JobQueue::getPeriodicTasks ( ) [final]

Return a map of task names to task definition maps.

A "task" is a fast periodic queue maintenance action. Mutually exclusive tasks must implement their own locking in the callback.

Each task value is an associative array with:

name : the name of the task
callback : a PHP callable that performs the task
period : the period in seconds corresponding to the task frequency

Returns:: Array

Definition at line 382 of file JobQueue.php.

References $name, as, and doGetPeriodicTasks().

JobQueue::getSize ( ) [final]

Get the number of available (unacquired) jobs in the queue.

Queue classes should use caching if they are any slower without memcached.

If caching is used, this number might be out of date for a minute.

Returns:: integer

Exceptions:

MWException

Definition at line 162 of file JobQueue.php.

References $res, doGetSize(), wfProfileIn(), and wfProfileOut().

JobQueue::getType ( ) [final]

Returns:: string Job type that this queue handles

Definition at line 107 of file JobQueue.php.

References type.

Referenced by JobQueueDB\getAllQueuedJobs().

JobQueue::getWiki ( ) [final]

Returns:: string Wiki ID

Definition at line 100 of file JobQueue.php.

References wiki.

JobQueue::isEmpty ( ) [final]

Quickly check if the queue is empty (has no available jobs).

Queue classes should use caching if they are any slower without memcached.

If caching is used, this might return false when there are actually no jobs. If pop() is called and returns false then it should correct the cache. Also, calling flushCaches() first prevents this. However, this affect is typically not distinguishable from the race condition between isEmpty() and pop().