[ Index ]

PHP Cross Reference of MediaWiki-1.24.0

title

Body

[close]

/includes/jobqueue/ -> JobQueueRedis.php (summary)

Redis-backed job queue code. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

Author: Aaron Schulz
File Size: 865 lines (27 kb)
Included or required:0 times
Referenced: 0 times
Includes or requires: 0 files

Defines 1 class

JobQueueRedis:: (34 methods):
  __construct()
  supportedOrders()
  optimalOrder()
  supportsDelayedJobs()
  doIsEmpty()
  doGetSize()
  doGetAcquiredCount()
  doGetDelayedCount()
  doGetAbandonedCount()
  doBatchPush()
  pushBlobs()
  doPop()
  popAndDeleteBlob()
  popAndAcquireBlob()
  doAck()
  doDeduplicateRootJob()
  doIsRootJobOldDuplicate()
  doDelete()
  getAllQueuedJobs()
  getAllDelayedJobs()
  getCoalesceLocationInternal()
  doGetSiblingQueuesWithJobs()
  doGetSiblingQueueSizes()
  getJobFromUidInternal()
  recyclePruneAndUndelayJobs()
  doGetPeriodicTasks()
  getNewJobFields()
  getJobFromFields()
  serialize()
  unserialize()
  getConnection()
  throwRedisException()
  getQueueKey()
  setTestingPrefix()


Class: JobQueueRedis  - X-Ref

Class to handle job queues stored in Redis

This is faster, less resource intensive, queue that JobQueueDB.
All data for a queue using this class is placed into one redis server.

There are eight main redis keys used to track jobs:
- l-unclaimed  : A list of job IDs used for ready unclaimed jobs
- z-claimed    : A sorted set of (job ID, UNIX timestamp as score) used for job retries
- z-abandoned  : A sorted set of (job ID, UNIX timestamp as score) used for broken jobs
- z-delayed    : A sorted set of (job ID, UNIX timestamp as score) used for delayed jobs
- h-idBySha1   : A hash of (SHA1 => job ID) for unclaimed jobs used for de-duplication
- h-sha1ById   : A hash of (job ID => SHA1) for unclaimed jobs used for de-duplication
- h-attempts   : A hash of (job ID => attempt count) used for job claiming/retries
- h-data       : A hash of (job ID => serialized blobs) for job storage
A job ID can be in only one of z-delayed, l-unclaimed, z-claimed, and z-abandoned.
If an ID appears in any of those lists, it should have a h-data entry for its ID.
If a job has a SHA1 de-duplication value and its ID is in l-unclaimed or z-delayed, then
there should be no other such jobs with that SHA1. Every h-idBySha1 entry has an h-sha1ById
entry and every h-sha1ById must refer to an ID that is l-unclaimed. If a job has its
ID in z-claimed or z-abandoned, then it must also have an h-attempts entry for its ID.

Additionally, "rootjob:* keys track "root jobs" used for additional de-duplication.
Aside from root job keys, all keys have no expiry, and are only removed when jobs are run.
All the keys are prefixed with the relevant wiki ID information.

This class requires Redis 2.6 as it makes use Lua scripts for fast atomic operations.
Additionally, it should be noted that redis has different persistence modes, such
as rdb snapshots, journaling, and no persistent. Appropriate configuration should be
made on the servers based on what queues are using it and what tolerance they have.

__construct( array $params )   X-Ref

param: array $params Possible keys:

supportedOrders()   X-Ref
No description

optimalOrder()   X-Ref
No description

supportsDelayedJobs()   X-Ref
No description

doIsEmpty()   X-Ref

return: bool

doGetSize()   X-Ref

return: int

doGetAcquiredCount()   X-Ref

return: int

doGetDelayedCount()   X-Ref

return: int

doGetAbandonedCount()   X-Ref

return: int

doBatchPush( array $jobs, $flags )   X-Ref

param: array $jobs
param: int $flags
return: void

pushBlobs( RedisConnRef $conn, array $items )   X-Ref

param: RedisConnRef $conn
param: array $items List of results from JobQueueRedis::getNewJobFields()
return: int Number of jobs inserted (duplicates are ignored)

doPop()   X-Ref

return: Job|bool

popAndDeleteBlob( RedisConnRef $conn )   X-Ref

param: RedisConnRef $conn
return: array Serialized string or false

popAndAcquireBlob( RedisConnRef $conn )   X-Ref

param: RedisConnRef $conn
return: array Serialized string or false

doAck( Job $job )   X-Ref

param: Job $job
return: Job|bool

doDeduplicateRootJob( Job $job )   X-Ref

param: Job $job
return: bool

doIsRootJobOldDuplicate( Job $job )   X-Ref

param: Job $job
return: bool

doDelete()   X-Ref

return: bool

getAllQueuedJobs()   X-Ref

return: Iterator

getAllDelayedJobs()   X-Ref
No description

getCoalesceLocationInternal()   X-Ref
No description

doGetSiblingQueuesWithJobs( array $types )   X-Ref
No description

doGetSiblingQueueSizes( array $types )   X-Ref
No description

getJobFromUidInternal( $uid, RedisConnRef $conn )   X-Ref
This function should not be called outside JobQueueRedis

param: string $uid
param: RedisConnRef $conn
return: Job|bool Returns false if the job does not exist

recyclePruneAndUndelayJobs()   X-Ref
Recycle or destroy any jobs that have been claimed for too long
and release any ready delayed jobs into the queue

return: int Number of jobs recycled/deleted/undelayed

doGetPeriodicTasks()   X-Ref

return: array

getNewJobFields( IJobSpecification $job )   X-Ref

param: IJobSpecification $job
return: array

getJobFromFields( array $fields )   X-Ref

param: array $fields
return: Job|bool

serialize( array $fields )   X-Ref

param: array $fields
return: string Serialized and possibly compressed version of $fields

unserialize( $blob )   X-Ref

param: string $blob
return: array|bool Unserialized version of $blob or false

getConnection()   X-Ref
Get a connection to the server that handles all sub-queues for this queue

return: RedisConnRef

throwRedisException( RedisConnRef $conn, $e )   X-Ref

param: RedisConnRef $conn
param: RedisException $e

getQueueKey( $prop, $type = null )   X-Ref

param: string $prop
param: string|null $type
return: string

setTestingPrefix( $key )   X-Ref

param: string $key
return: void



Generated: Fri Nov 28 14:03:12 2014 Cross-referenced by PHPXref 0.7.1