MediaWiki
REL1_23
|
Class to handle enqueueing and running of background jobs for federated queues. More...
Public Member Functions | |
getAllDelayedJobs () | |
Get an iterator to traverse over all delayed jobs in this queue. | |
getAllQueuedJobs () | |
Get an iterator to traverse over all available jobs in this queue. | |
getCoalesceLocationInternal () | |
Do not use this function outside of JobQueue/JobQueueGroup. | |
setTestingPrefix ($key) | |
Namespace the queue with a key to isolate it for testing. | |
Public Attributes | |
const | CACHE_TTL_LONG = 300 |
const | CACHE_TTL_SHORT = 30 |
Protected Member Functions | |
__construct (array $params) | |
include: | |
doAck (Job $job) | |
doBatchPush (array $jobs, $flags) | |
doDeduplicateRootJob (Job $job) | |
doDelete () | |
doFlushCaches () | |
doGetAbandonedCount () | |
doGetAcquiredCount () | |
doGetDelayedCount () | |
doGetPeriodicTasks () | |
doGetSiblingQueueSizes (array $types) | |
doGetSiblingQueuesWithJobs (array $types) | |
doGetSize () | |
doIsEmpty () | |
doIsRootJobOldDuplicate (Job $job) | |
doPop () | |
doWaitForBackups () | |
getCrossPartitionSum ($type, $method) | |
optimalOrder () | |
Get the default queue order to use if configuration does not specify one. | |
supportedOrders () | |
Get the allowed queue orders for configuration validation. | |
supportsDelayedJobs () | |
Find out if delayed jobs are supported for configuration validation. | |
throwErrorIfAllPartitionsDown ($down) | |
Throw an error if no partitions available. | |
tryJobInsertions (array $jobs, HashRing &$partitionRing, $flags) | |
Protected Attributes | |
BagOStuff | $cache |
* | |
int | $maxPartitionsTry |
Maximum number of partitions to try *. | |
array | $partitionMap = array() |
(partition name => weight) reverse sorted by weight * | |
HashRing | $partitionPushRing |
* | |
array | $partitionQueues = array() |
(partition name => JobQueue) reverse sorted by weight * | |
Private Member Functions | |
getCacheKey ($property) |
Class to handle enqueueing and running of background jobs for federated queues.
This class allows for queues to be partitioned into smaller queues. A partition is defined by the configuration for a JobQueue instance. For example, one can set $wgJobTypeConf['refreshLinks'] to point to a JobQueueFederated instance, which itself would consist of three JobQueueRedis instances, each using their own redis server. This would allow for the jobs to be split (evenly or based on weights) accross multiple servers if a single server becomes impractical or expensive. Different JobQueue classes can be mixed.
The basic queue configuration (e.g. "order", "claimTTL") of a federated queue is inherited by the partition queues. Additional configuration defines what section each wiki is in, what partition queues each section uses (and their weight), and the JobQueue configuration for each partition. Some sections might only need a single queue partition, like the sections for groups of small wikis.
If used for performance, then $wgMainCacheType should be set to memcached/redis. Note that "fifo" cannot be used for the ordering, since the data is distributed. One can still use "timestamp" instead, as in "roughly timestamp ordered". Also, queue classes used by this should ignore down servers (with TTL) to avoid slowness.
Definition at line 49 of file JobQueueFederated.php.
JobQueueFederated::__construct | ( | array $ | params | ) | [protected] |
include:
array | $params |
MWException |
Reimplemented from JobQueue.
Definition at line 83 of file JobQueueFederated.php.
JobQueueFederated::doAck | ( | Job $ | job | ) | [protected] |
Job | $job |
Reimplemented from JobQueue.
Definition at line 346 of file JobQueueFederated.php.
JobQueueFederated::doBatchPush | ( | array $ | jobs, |
$ | flags | ||
) | [protected] |
array | $jobs | |
$flags |
Reimplemented from JobQueue.
Definition at line 209 of file JobQueueFederated.php.
JobQueueFederated::doDeduplicateRootJob | ( | Job $ | job | ) | [protected] |
Job | $job |
MWException |
Reimplemented from JobQueue.
Definition at line 368 of file JobQueueFederated.php.
JobQueueFederated::doDelete | ( | ) | [protected] |
MWException |
Reimplemented from JobQueue.
Definition at line 382 of file JobQueueFederated.php.
JobQueueFederated::doFlushCaches | ( | ) | [protected] |
Reimplemented from JobQueue.
Definition at line 423 of file JobQueueFederated.php.
JobQueueFederated::doGetAbandonedCount | ( | ) | [protected] |
Reimplemented from JobQueue.
Definition at line 176 of file JobQueueFederated.php.
References getCrossPartitionSum().
JobQueueFederated::doGetAcquiredCount | ( | ) | [protected] |
Reimplemented from JobQueue.
Definition at line 168 of file JobQueueFederated.php.
References getCrossPartitionSum().
JobQueueFederated::doGetDelayedCount | ( | ) | [protected] |
Reimplemented from JobQueue.
Definition at line 172 of file JobQueueFederated.php.
References getCrossPartitionSum().
JobQueueFederated::doGetPeriodicTasks | ( | ) | [protected] |
Reimplemented from JobQueue.
Definition at line 411 of file JobQueueFederated.php.
JobQueueFederated::doGetSiblingQueueSizes | ( | array $ | types | ) | [protected] |
array | $types | List of queues types |
Reimplemented from JobQueue.
Definition at line 495 of file JobQueueFederated.php.
JobQueueFederated::doGetSiblingQueuesWithJobs | ( | array $ | types | ) | [protected] |
array | $types | List of queues types |
Reimplemented from JobQueue.
Definition at line 469 of file JobQueueFederated.php.
JobQueueFederated::doGetSize | ( | ) | [protected] |
Reimplemented from JobQueue.
Definition at line 164 of file JobQueueFederated.php.
JobQueueFederated::doIsEmpty | ( | ) | [protected] |
Reimplemented from JobQueue.
Definition at line 138 of file JobQueueFederated.php.
JobQueueFederated::doIsRootJobOldDuplicate | ( | Job $ | job | ) | [protected] |
Job | $job |
Reimplemented from JobQueue.
Definition at line 354 of file JobQueueFederated.php.
JobQueueFederated::doPop | ( | ) | [protected] |
Reimplemented from JobQueue.
Definition at line 305 of file JobQueueFederated.php.
JobQueueFederated::doWaitForBackups | ( | ) | [protected] |
Reimplemented from JobQueue.
Definition at line 397 of file JobQueueFederated.php.
Get an iterator to traverse over all delayed jobs in this queue.
Note: results may be stale if the queue is concurrently modified.
JobQueueError |
Reimplemented from JobQueue.
Definition at line 453 of file JobQueueFederated.php.
Get an iterator to traverse over all available jobs in this queue.
This does not include jobs that are currently acquired or delayed. Note: results may be stale if the queue is concurrently modified.
JobQueueError |
Reimplemented from JobQueue.
Definition at line 442 of file JobQueueFederated.php.
JobQueueFederated::getCacheKey | ( | $ | property | ) | [private] |
$property |
Definition at line 543 of file JobQueueFederated.php.
References $property, list, type, wfForeignMemcKey(), and wfSplitWikiID().
Do not use this function outside of JobQueue/JobQueueGroup.
Reimplemented from JobQueue.
Definition at line 464 of file JobQueueFederated.php.
JobQueueFederated::getCrossPartitionSum | ( | $ | type, |
$ | method | ||
) | [protected] |
string | $type | |
string | $method |
Definition at line 185 of file JobQueueFederated.php.
Referenced by doGetAbandonedCount(), doGetAcquiredCount(), and doGetDelayedCount().
JobQueueFederated::optimalOrder | ( | ) | [protected] |
Get the default queue order to use if configuration does not specify one.
Reimplemented from JobQueue.
Definition at line 130 of file JobQueueFederated.php.
References array().
JobQueueFederated::setTestingPrefix | ( | $ | key | ) |
Namespace the queue with a key to isolate it for testing.
string | $key |
MWException |
Reimplemented from JobQueue.
Definition at line 532 of file JobQueueFederated.php.
JobQueueFederated::supportedOrders | ( | ) | [protected] |
Get the allowed queue orders for configuration validation.
Reimplemented from JobQueue.
Definition at line 125 of file JobQueueFederated.php.
JobQueueFederated::supportsDelayedJobs | ( | ) | [protected] |
Find out if delayed jobs are supported for configuration validation.
Reimplemented from JobQueue.
Definition at line 134 of file JobQueueFederated.php.
JobQueueFederated::throwErrorIfAllPartitionsDown | ( | $ | down | ) | [protected] |
Throw an error if no partitions available.
int | $down | The number of up partitions down |
JobQueueError |
Definition at line 526 of file JobQueueFederated.php.
JobQueueFederated::tryJobInsertions | ( | array $ | jobs, |
HashRing &$ | partitionRing, | ||
$ | flags | ||
) | [protected] |
array | $jobs | |
HashRing | $partitionRing | |
int | $flags |
JobQueueError |
Definition at line 233 of file JobQueueFederated.php.
*
Definition at line 56 of file JobQueueFederated.php.
int JobQueueFederated::$maxPartitionsTry [protected] |
Maximum number of partitions to try *.
Definition at line 58 of file JobQueueFederated.php.
(partition name => weight) reverse sorted by weight *
Definition at line 50 of file JobQueueFederated.php.
HashRing JobQueueFederated::$partitionPushRing [protected] |
*
Definition at line 54 of file JobQueueFederated.php.
(partition name => JobQueue) reverse sorted by weight *
Definition at line 52 of file JobQueueFederated.php.
const JobQueueFederated::CACHE_TTL_LONG = 300 |
Definition at line 61 of file JobQueueFederated.php.
const JobQueueFederated::CACHE_TTL_SHORT = 30 |
Definition at line 60 of file JobQueueFederated.php.