The ceilometer.storage.impl_hbase Module

Openstack Ceilometer HBase storage backend

Note

This driver is designed to enable Ceilometer store its data in HBase. The implementation is using HBase Thrift interface so it’s necessary to have the HBase Thrift server installed and started: (https://ccp.cloudera.com/display/CDHDOC/HBase+Installation)

This driver has been tested against HBase 0.92.1/CDH 4.1.1, HBase 0.94.4/HDP 1.2 and HBase 0.94.5/Apache. Versions earlier than 0.92.1 are not supported due to feature incompatibility.

Due to limitations of HBase the driver implements its own data aggregations which may harm its performance. It is likely that the performance could be improved if co-processors were used, however at the moment the co-processor support is not exposed through Thrift API.

The following four tables are expected to exist in HBase:

create ‘project’, {NAME=>’f’} create ‘user’, {NAME=>’f’} create ‘resource’, {NAME=>’f’} create ‘meter’, {NAME=>’f’}

The driver is using HappyBase which is a wrapper library used to interact with HBase via Thrift protocol: http://happybase.readthedocs.org/en/latest/index.html#

class ceilometer.storage.impl_hbase.Connection(conf)

Bases: ceilometer.storage.base.Connection

HBase connection.

clear()

get_event_interval(event_filter)

Return the min and max timestamps from samples, using the event_filter to limit the samples seen.

( datetime.datetime(), datetime.datetime() )

get_meter_statistics(event_filter, period=None)

Return a dictionary containing meter statistics. described by the query parameters.

The filter must have a meter value set.

{ ‘min’:

‘max’: ‘avg’: ‘sum’: ‘count’: ‘period’: ‘period_start’: ‘period_end’: ‘duration’: ‘duration_start’: ‘duration_end’: }

Due to HBase limitations the aggregations are implemented in the driver itself, therefore this method will be quite slow because of all the Thrift traffic it is going to create.

get_meters(user=None, project=None, resource=None, source=None, metaquery={})

Return an iterable of dictionaries containing meter information.

{ ‘name’: name of the meter,

‘type’: type of the meter (guage, counter), ‘unit’: unit of the meter, ‘resource_id’: UUID of the resource, ‘project_id’: UUID of project owning the resource, ‘user_id’: UUID of user owning the resource, }

Parameters:

• user – Optional ID for user that owns the resource.
• project – Optional ID for project that owns the resource.
• resource – Optional resource filter.
• source – Optional source filter.
• metaquery – Optional dict with metadata to match on.

get_projects(source=None)

Return an iterable of project id strings.

Parameters:source – Optional source filter.

get_resources(user=None, project=None, source=None, start_timestamp=None, end_timestamp=None, metaquery={})

Return an iterable of dictionaries containing resource information.

{ ‘resource_id’: UUID of the resource,

‘project_id’: UUID of project owning the resource, ‘user_id’: UUID of user owning the resource, ‘timestamp’: UTC datetime of last update to the resource, ‘metadata’: most current metadata for the resource, ‘meter’: list of the meters reporting data for the resource, }

Parameters:

• user – Optional ID for user that owns the resource.
• project – Optional ID for project that owns the resource.
• source – Optional source filter.
• start_timestamp – Optional modified timestamp start range.
• end_timestamp – Optional modified timestamp end range.

get_samples(event_filter)

Return an iterable of samples as created by ceilometer.meter.meter_message_from_counter().

get_users(source=None)

Return an iterable of user id strings.

Parameters:source – Optional source filter.

get_volume_max(event_filter)

Return the maximum of the volume field for the samples described by the query parameters.

get_volume_sum(event_filter)

Return the sum of the volume field for the samples described by the query parameters.

record_metering_data(data)

Write the data to the backend storage system.

Parameters:data – a dictionary such as returned by ceilometer.meter.meter_message_from_counter

upgrade(version=None)

class ceilometer.storage.impl_hbase.HBaseStorage

Bases: ceilometer.storage.base.StorageEngine

Put the data into a HBase database

Collections:

• user - { _id: user id

  source: [ array of source ids reporting for the user ] }

• project - { _id: project id

  source: [ array of source ids reporting for the project ] }

• meter - the raw incoming data

• resource - the metadata for resources - { _id: uuid of resource,

  metadata: metadata dictionaries timestamp: datetime of last update user_id: uuid project_id: uuid meter: [ array of {counter_name: string, counter_type: string} ]

  }

static get_connection(conf)

Return a Connection instance based on the configuration settings.

register_opts(conf)

Register any configuration options used by this engine.