Version of PoolCounter that uses Redis. More...

Inheritance diagram for PoolCounterRedis:

Collaboration diagram for PoolCounterRedis:

Public Member Functions
	__construct ($conf, $type, $key)

	acquireForAnyone ()

	acquireForMe ()

	release ()

Public Member Functions inherited from PoolCounter
	acquireForAnyone ()
	I want to do this task, but if anyone else does it instead, it's also fine for me. More...

	acquireForMe ()
	I want to do this task and I need to do it myself. More...

	getKey ()

	release ()
	I have successfully finished my task. More...

Static Public Member Functions
static	releaseAll ()
	Try to make sure that locks get released (even with exceptions and fatals) More...

Static Public Member Functions inherited from PoolCounter
static	factory ($type, $key)
	Create a Pool counter. More...

Public Attributes
const	AWAKE_ALL = 2

const	AWAKE_ONE = 1

Public Attributes inherited from PoolCounter
const	DONE = 3

const	ERROR = -1

const	LOCK_HELD = -5

const	LOCKED = 1

const	NOT_LOCKED = -2

const	QUEUE_FULL = -3

const	RELEASED = 2

const	TIMEOUT = -4

Protected Member Functions
	getConnection ()

	getSlotListKey ()

	getSlotRTimeSetKey ()

	getWaitSetKey ()

	getWakeupListKey ()

	initAndPopPoolSlotList (RedisConnRef $conn, $now)

	registerAcquisitionTime (RedisConnRef $conn, $slot, $now)

	waitForSlotOrNotif ($doWakeup)

Protected Member Functions inherited from PoolCounter
	__construct ($conf, $type, $key)

	hashKeyIntoSlots ($type, $key, $slots)
	Given a key (any string) and the number of lots, returns a slot key (a prefix with a suffix integer from the [0..($slots-1)] range). More...

	onAcquire ()
	Update any lock tracking information when the lock is acquired. More...

	onRelease ()
	Update any lock tracking information when the lock is released. More...

	precheckAcquire ()
	Checks that the lock request is sane. More...

Protected Attributes
RedisConnRef	$conn

string	$keySha1
	SHA-1 of the key. More...

int	$lockTTL
	TTL for locks to expire (work should finish in this time) More...

int	$onRelease
	AWAKE_* constant. More...

RedisConnectionPool	$pool

HashRing	$ring

array	$serversByLabel
	(server label => host) map More...

string	$session
	Unique string to identify this process. More...

string	$slot
	Pool slot value. More...

int	$slotTime
	UNIX timestamp. More...

Protected Attributes inherited from PoolCounter
string	$key
	All workers with the same key share the lock. More...

int	$maxqueue
	If this number of workers are already working/waiting, fail instead of wait. More...

int	$slots = 0
	Maximum number of workers working on this task type, regardless of key. More...

float	$timeout
	Maximum time in seconds to wait for the lock. More...

int	$workers
	Maximum number of workers working on tasks with the same key simultaneously. More...

Static Protected Attributes
static PoolCounterRedis[]	$active = null
	List of active PoolCounterRedis objects in this script. More...

Detailed Description

Version of PoolCounter that uses Redis.

There are four main redis keys used to track each pool counter key:

poolcounter:l-slots-* : A list of available slot IDs for a pool.
poolcounter:z-renewtime-* : A sorted set of (slot ID, UNIX timestamp as score) used for tracking the next time a slot should be released. This is -1 when a slot is created, and is set when released (expired), locked, and unlocked.
poolcounter:z-wait-* : A sorted set of (slot ID, UNIX timestamp as score) used for tracking waiting processes (and wait time).
poolcounter:l-wakeup-* : A list pushed to for the sake of waking up processes when a any process in the pool finishes (lasts for 1ms). For a given pool key, all the redis keys start off non-existing and are deleted if not used for a while to prevent garbage from building up on the server. They are atomically re-initialized as needed. The "z-renewtime" key is used for detecting sessions which got slots but then disappeared. Stale entries from there have their timestamp updated and the corresponding slots freed up. The "z-wait" key is used for detecting processes registered as waiting but that disappeared. Stale entries from there are deleted and the corresponding slots are freed up. The worker count is included in all the redis key names as it does not vary within each $wgPoolCounterConf type and doing so handles configuration changes.

This class requires Redis 2.6 as it makes use Lua scripts for fast atomic operations. Also this should be on a server plenty of RAM for the working set to avoid evictions. Evictions could temporarily allow wait queues to double in size or temporarily cause pools to appear as full when they are not. Using volatile-ttl and bumping memory-samples in redis.conf can be helpful otherwise.