Package pgq :: Module consumer :: Class SerialConsumer

Class SerialConsumer

                 object --+        
                          |        
skytools.scripting.DBScript --+    
                              |    
                       Consumer --+
                                  |
                                 SerialConsumer

Known Subclasses:: londiste.playback.Replicator

Consumer that applies batches sequentially in second database.

Requirements:

Whole batch in one TX.
Must not use retry queue.

Features:

Can detect if several batches are already applied to dest db.
If some ticks are lost. allows to seek back on queue. Whether it succeeds, depends on pgq configuration.

Instance Methods

__init__(self, service_name, db_name, remote_db, args)
Initialize new consumer.

source code

attach(self)
Attach consumer to interesting queues.

source code

clean_completed_tick(self)

source code

detach(self)
If detaching, also clean completed tick table on dest.

source code

dst_reset(self)

source code

init_optparse(self, parser=None)
Initialize a OptionParser() instance that will be used to parse command line arguments.

source code

is_batch_done(self, dst_curs)
Helper function to keep track of last successful batch in external database.

source code

process_batch(self, db, batch_id, event_list)
Process all events in batch.

source code

process_remote_batch(self, db, batch_id, event_list, dst_db)

source code

rewind(self)

source code

set_batch_done(self, dst_curs)
Helper function to set last successful batch in external database.

source code

startup(self)
Will be called just before entering main loop.

source code

Inherited from Consumer: get_batch_info, process_event, register_consumer, stat_end, stat_start, unregister_consumer, work

Inherited from skytools.scripting.DBScript: close_database, get_database, hook_sighup, hook_sigint, reload, reset, run, run_once, send_signal, send_stats, set_single_loop, start, stat_add, stat_increase, stat_put, stop

Inherited from object: __delattr__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __str__

Class Variables
Inherited from `skytools.scripting.DBScript`: `cf`, `job_name`, `log`, `service_name`

Properties
Inherited from `object`: `__class__`

Method Details

init(self, service_name, db_name, remote_db, args)
(Constructor)

source code

Initialize new consumer.

Overrides: Consumer.__init__: (inherited documentation)

attach(self)

source code

Attach consumer to interesting queues.

Overrides: Consumer.attach: (inherited documentation)

detach(self)

source code

If detaching, also clean completed tick table on dest.

Overrides: Consumer.detach

init_optparse(self, parser=None)

source code

Initialize a OptionParser() instance that will be used to parse command line arguments.

Note that it can be overrided both directions - either DBScript will initialize a instance and passes to user code or user can initialize and then pass to DBScript.init_optparse().

Returns:: initialized OptionParser() instance.
Overrides: skytools.scripting.DBScript.init_optparse: (inherited documentation)

process_batch(self, db, batch_id, event_list)

source code

Process all events in batch.

Overrides: Consumer.process_batch

startup(self)

source code

Will be called just before entering main loop.

In case of daemon, if will be called in same process as work(), unlike __init__().

Overrides: skytools.scripting.DBScript.startup: (inherited documentation)

Class SerialConsumer

__init__(self, service_name, db_name, remote_db, args) (Constructor)

attach(self)

detach(self)

init_optparse(self, parser=None)

process_batch(self, db, batch_id, event_list)

startup(self)

init(self, service_name, db_name, remote_db, args)
(Constructor)