Developer's Guide
The zenhub daemon (aka the Hub or ZenHub) is a single-threaded and asynchronous daemon that provides the following features:
Connections between daemons and the ZODB for persistent object management (for example, configuration loading). Writes to the ZODB are synchronous operations.
Connections between daemons and the MySQL event database for events and event management. Writes to MySQL are synchronous operations.
Connections between daemons and performance data in RRD files
Pluggable Daemon Services
User-interactive RRD graph fetching (such as renderserver functionality)
Loading configuration
The Hub (as of Zenoss version 2.3) can be split out some of its tasks by creating workers (a configuration file option). Requests from collectors are farmed out to the worker processes to spread out some of the load.
Note
Propagating configuration changes and fetching RRD Data is not pushed through workers! This means that large configuration downloads will still affect the user experience. Some sort of caching on the daemon's side may be necessary for large sites.
The zenhub daemon manages updates to the object database (ZODB) to any daemons that connect to zenhub (in practice this means all Zenoss daemons). The Hub watches for changes to the ZODB database (for example, the use of the commit() function) and initiates change notifications to any affected daemons. zenhub also provides daemons access to the object database for loading configuration items and posting events.
Another management function that zenhub provides is the ability to send notifications (ie Zenoss events). An event will be provided from the daemon to the Hub which then stores the event in the event database (ie a MySQL table) and then the event is processed according to any mappings that match the event. In this way an event generated by an error condition can be cleared by another event.
Each daemon should post an event when it is shutdown, so that the console is kept informed of intentional shutdowns. However, these events should be cleared by matching start events. Start/shutdown events should only be sent when the server is daemon-ized.
Each daemon should post a periodic Heartbeat event. If a heartbeat event is not updated the Zenoss GUI will indicate a problem with the daemon. Ideally, a daemon only sends a heartbeat event after each successful operating cycle (after each performance data collection). It is not acceptable to just post events in a separate thread or timer unless that thread also does some minimal testing for internal status and health.
If the daemon cannot talk to the Hub (zenhub is down) then events are queued up. When communications are restored the queued events are then delivered.
To implement these features, zenhub has a collection of Services that it is willing to provide to other daemons. A daemon will connect and request a particular Service. ZenHub will create that Service, and send future object change notices to the Service, which in turn can decide how best to notify the daemon. Some daemons, such as zenping, have a very simple configuration that doesn't change very often. Others, such as the zenperfsnmp, have a much more complex configuration that must be kept up-to-date with model changes.
Each Service is implemented as a class that zenhub can import. Using Twisted's Perspective Broker (PB) facilities, a daemon can request that the Hub perform some action (ie a class method) and return the results to the daemon, and vice versa. In other words, the Service acts as the interface between the daemon and the Hub. The services directory in a ZenPack directory structure is where the Service class is kept.