www.espertech.comDocumentation
Esper engine configuration is entirely optional. Esper has a very small number of configuration parameters that can be used to simplify event pattern and EPL statements, and to tune the engine behavior to specific requirements. The Esper engine works out-of-the-box without configuration.
An application can supply configuration at the time of engine allocation using the Configuration
class, and can also use XML files to hold configuration. Configuration can be changed at runtime
via the ConfigurationOperations
interface available from EPAdministrator
via the getConfiguration
method.
The difference between using a Configuration
object and the ConfigurationOperations
interface is that for the latter, all configuration including event types added through that interface are considered runtime configurations.
This means they will be discarded when calling the initialize
method on an EPServiceProvider
instance.
An instance of com.espertech.esper.client.Configuration
represents all configuration parameters. The Configuration
is used to build an EPServiceProvider
, which provides the administrative and runtime interfaces for an Esper engine instance.
You may obtain a Configuration
instance by instantiating it directly and adding or setting values on it. The Configuration
instance is then passed to EPServiceProviderManager
to obtain a configured Esper engine.
Configuration configuration = new Configuration(); configuration.addEventType("PriceLimit", PriceLimit.class.getName()); configuration.addEventType("StockTick", StockTick.class.getName()); configuration.addImport("org.mycompany.mypackage.MyUtility"); configuration.addImport("org.mycompany.util.*"); EPServiceProvider epService = EPServiceProviderManager.getProvider("sample", configuration);
Note that Configuration
is meant only as an initialization-time object. The Esper engine represented by an EPServiceProvider
does not retain any association back to the Configuration
.
The ConfigurationOperations
interface provides runtime configuration options as further described in Section 15.3.8, “Runtime Configuration”. Through this interface applications can, for example, add new event types at runtime and then create new statements
that rely on the additional configuration. The getConfiguration
method on EPAdministrator
allows access to ConfigurationOperations
.
select * from OrderEvent
The XML configuration for defining the Java packages that contain Java event classes is:
<event-type-auto-name package-name="com.mycompany.order.event"/>
The same configuration but using the Configuration
class:
Configuration config = new Configuration(); config.addEventTypeAutoName("com.mycompany.order.event"); // ... or ... config.addEventTypeAutoName(MyEvent.getPackage().getName());
every StockTick(symbol='IBM')"
every com.espertech.esper.example.stockticker.event.StockTick(symbol='IBM')
<event-type name="StockTick" class="com.espertech.esper.example.stockticker.event.StockTick"/>
<event-type name="StockTick" class="com.espertech.esper.example.stockticker.event.StockTickEvent"> <legacy-type accessor-style="javabean" code-generation="enabled"> <method-property name="price" accessor-method="getCurrentPrice" /> <field-property name="volume" accessor-field="volumeField" /> </legacy-type> </event-type>
select avg(price * volume) from StockTick
As with all configuration options, the API can also be used:
Configuration configuration = new Configuration(); ConfigurationEventTypeLegacy legacyDef = new ConfigurationEventTypeLegacy(); legacyDef.addMethodProperty("price", "getCurrentPrice"); legacyDef.addFieldProperty("volume", "volumeField"); config.addEventType("StockTick", StockTickEvent.class.getName(), legacyDef);
<event-type name="MyLegacyEvent" class="com.mycompany.package.MyLegacyEventClass"> <legacy-type accessor-style="javabean" code-generation="disabled" /> </event-type>
The sample below shows how to configure this option via the API.
Configuration configuration = new Configuration(); ConfigurationEventTypeLegacy legacyDef = new ConfigurationEventTypeLegacy(); legacyDef.setCodeGeneration(ConfigurationEventTypeLegacy.CodeGeneration.DISABLED); config.addEventType("MyLegacyEvent", MyLegacyEventClass.class.getName(), legacyDef);
The engine can process java.util.Map
events via the sendEvent(Map map, String eventTypeName)
method on the EPRuntime
interface. Entries in the Map represent event properties. Keys must be of type java.util.String
for the engine to be able to look up event property names in pattern or EPL statements. Values can be of any type. JavaBean-style objects as values in a Map
can be processed by the engine, and strongly-typed nested maps are also supported. Please see the Chapter 2, Event Representations section for details on how to use Map
events with the engine.
Via configuration you can provide an event type name for Map
events for use in statements, and the event property names and types enabling the engine to validate properties in statements.
The below snippet of XML configuration configures an event named MyMapEvent
.
<event-type name="MyMapEvent"> <java-util-map> <map-property name="carId" class="int"/> <map-property name="carType" class="string"/> <map-property name="assembly" class="com.mycompany.Assembly"/> </java-util-map> </event-type>
This configuration defines the carId
property of MyMapEvent
events to be of type int
, and the carType
property to be of type java.util.String
. The assembly
property of the Map event will contain instances of com.mycompany.Assembly
for the engine to query.
The valid types for the class
attribute are listed in Section 16.5, “Type Names”. In addition, any fully-qualified Java class name that can be resolved via Class.forName
is allowed.
You can also use the configuration API to configure Map
event types, as the short code snippet below demonstrates:
Map<String, Object> properties = new Map<String, Object>(); properties.put("carId", "int"); properties.put("carType", "string"); properties.put("assembly", Assembly.class.getName()); Configuration configuration = new Configuration(); configuration.addEventType("MyMapEvent", properties);
For strongly-typed nested maps (maps-within-maps), the configuration API method addEventType
can also used to define the nested types. The XML configuration does not provide the capability to configure nested maps.
Finally, here is a sample EPL statement that uses the configured MyMapEvent
map event. This statement uses the chassisTag
and numParts
properties of Assembly
objects in each map.
select carType, assembly.chassisTag, count(assembly.numParts) from MyMapEvent.win:time(60 sec)
A Map event type may also become a subtype of one or more supertypes that must also be Map event types. The java-util-map
element provides the optional attribute supertype-names
that accepts a comma-separated list of names of Map event types that are supertypes to the type:
<event-type name="AccountUpdate"> <java-util-map supertype-names="BaseUpdate, AccountEvent"> ...
For initialization time configuration, the addMapSuperType
method can be used to add Map hierarchy information. For runtime configuration, pass the supertype names to the addEventType
method in ConfigurationOperations
.
A Map event type may declare a start and end timestamp property name. The XML shown next instructs the engine that the startts
property carries the event start timestamp and the endts
property carries the event end timestamp:
<event-type name="AccountUpdate"> <java-util-map start-timestamp-property-name="startts" end-timestamp-property-name="endts"> ...
The engine can process Object-array (Object[]
) events via the sendEvent(Object[] array, String eventTypeName)
method on the EPRuntime
interface. Elements in the Object array represent event properties. Values can be of any type. JavaBean-style objects as values in an Object[]
can be processed by the engine, and strongly-typed nested map or object-array event types are also supported. Please see the Chapter 2, Event Representations section for details on how to use Object[]
events with the engine.
Via configuration you can provide an event type name for Object[]
events for use in statements, and the event property names and types enabling the engine to validate properties in statements.
The below snippet of XML configuration configures an event named MyObjectArrayEvent
.
<event-type name="MyObjectArrayEvent"> <objectarray> <objectarray-property name="carId" class="int"/> <objectarray-property name="carType" class="string"/> <objectarray-property name="assembly" class="com.mycompany.Assembly"/> </objectarray> </event-type>
This configuration defines the carId
property of MyObjectArrayEvent
events to be of type int
and in the object array first element ([0]
). The carType
property to be of type java.util.String
is expected in the second array element ([1]
) . The assembly
property of the object array event will contain instances of com.mycompany.Assembly
for the engine to query in element two ([2]
).
Note that the engine does not verify the length and property values of object array events when your application sends object-array events into the engine. For the example above, the proper object array would look as follows: new Object[] {carId, carType, assembly}
.
The valid types for the class
attribute are listed in Section 16.5, “Type Names”. In addition, any fully-qualified Java class name that can be resolved via Class.forName
is allowed.
You can also use the configuration API to configure Object[]
event types, as the short code snippet below demonstrates:
String[] propertyNames = {"carId", "carType", "assembly"}; Object[] propertyTypes = {int.class, String.class, Assembly.class}; Configuration configuration = new Configuration(); configuration.addEventType("MyObjectArrayEvent", propertyNames, propertyTypes);
Finally, here is a sample EPL statement that uses the configured MyObjectArrayEvent
object-array event. This statement uses the chassisTag
and numParts
properties of Assembly
objects.
select carType, assembly.chassisTag, count(assembly.numParts) from MyObjectArrayEvent.win:time(60 sec)
An Object-array event type may also become a subtype of one supertype that must also be an Object-array event type. The objectarray
element provides the optional attribute supertype-names
that accepts a single name of an Object-array event type that is the supertype to the type:
<event-type name="AccountUpdate"> <objectarray supertype-names="BaseUpdate"> ...
An Object-array event type may declare a start and end timestamp property name. The XML shown next instructs the engine that the startts
property carries the event start timestamp and the endts
property carries the event end timestamp:
<event-type name="AccountUpdate"> <objectarray start-timestamp-property-name="startts" end-timestamp-property-name="endts"> ...
Via this configuration item the Esper engine can natively process org.w3c.dom.Node
instances, i.e. XML document object model (DOM) nodes.
Please see the Chapter 2, Event Representations section for details on how to use Node
events with the engine.
Esper allows configuring XPath expressions as event properties. You can specify arbitrary XPath functions or expressions and provide a property name by which their result values will be available for use in expressions.
For XML documents that follow a XML schema, Esper can load and interrogate your schema and validate event property names and types against the schema information.
Nested, mapped and indexed event properties are also supported in expressions against org.w3c.dom.Node
events. Thus XML trees can conveniently be
interrogated using the existing event property syntax for querying JavaBean objects, JavaBean object graphs or java.util.Map
events.
In the simplest form, the Esper engine only requires a configuration entry containing the root element name and the event type name in order to process org.w3c.dom.Node
events:
<event-type name="MyXMLNodeEvent"> <xml-dom root-element-name="myevent" /> </event-type>
You can also use the configuration API to configure XML event types, as the short example below demonstrates.
In fact, all configuration options available through XML configuration can
also be provided via setter methods on the ConfigurationEventTypeXMLDOM
class.
Configuration configuration = new Configuration(); ConfigurationEventTypeXMLDOM desc = new ConfigurationEventTypeXMLDOM(); desc.setRootElementName("myevent"); desc.addXPathProperty("name1", "/element/@attribute", XPathConstants.STRING); desc.addXPathProperty("name2", "/element/subelement", XPathConstants.NUMBER); configuration.addEventType("MyXMLNodeEvent", desc);
The next example presents configuration options in a sample configuration entry.
<event-type name="AutoIdRFIDEvent"> <xml-dom root-element-name="Sensor" schema-resource="data/AutoIdPmlCore.xsd" default-namespace="urn:autoid:specification:interchange:PMLCore:xml:schema:1"> <namespace-prefix prefix="pmlcore" namespace="urn:autoid:specification:interchange:PMLCore:xml:schema:1"/> <xpath-property property-name="countTags" xpath="count(/pmlcore:Sensor/pmlcore:Observation/pmlcore:Tag)" type="number"/> </xml-dom> </event-type>
This example configures an event property named countTags
whose value is computed by an XPath expression. The namespace prefixes
and default namespace are for use with XPath expressions and must also be made known to the engine in order for the engine to compile XPath expressions.
Via the schema-resource
attribute we instruct the engine to load a schema file. You may also use schema-text
instead to
provide the actual text of the schema.
Here is an example EPL statement using the configured event type named AutoIdRFIDEvent
.
select ID, countTags from AutoIdRFIDEvent.win:time(30 sec)
By configuring a schema file for the engine to load, the engine performs these additional services:
desc.addXPathProperty("name", "/element/sub", XPathConstants.STRING, "long");
The equivalent configuration XML is:
<xpath-property property-name="name" xpath="/element/sub" type="string" cast="long"/>
See Section 16.5, “Type Names” for a list of cast-to type names.
select request, request.symbol from GetQuote
The setting is available in XML via the attribute resolve-properties-absolute
.
The configuration API provides the above settings as shown here in a sample code:
ConfigurationEventTypeXMLDOM desc = new ConfigurationEventTypeXMLDOM(); desc.setRootElementName("GetQuote"); desc.setDefaultNamespace("http://services.samples/xsd"); desc.setRootElementNamespace("http://services.samples/xsd"); desc.addNamespacePrefix("m0", "http://services.samples/xsd"); desc.setResolvePropertiesAbsolute(false); configuration.addEventType("GetQuote", desc);
This example shows the API to set this configuration.
ConfigurationEventTypeXMLDOM desc = new ConfigurationEventTypeXMLDOM(); desc.setXPathFunctionResolver(MyXPathFunctionResolver.class.getName()); desc.setXPathVariableResolver(MyXPathVariableResolver.class.getName());
ConfigurationEventTypeXMLDOM desc = new ConfigurationEventTypeXMLDOM(); desc.setAutoFragment(false);
This configuration setting, when set to false, instructs an EventSender
to not validate.
ConfigurationEventTypeXMLDOM desc = new ConfigurationEventTypeXMLDOM(); desc.setEventSenderValidatesRoot(false);
As part of the extension API plug-in event representations allows an application to create new event types and event instances based on information available elsewhere. Please see Section 18.8, “Event Type And Event Object” for details.
The configuration examples shown next use the configuration API to select settings. All options are also configurable via XML, please refer to the sample configuration XML in file esper.sample.cfg.xml
.
Use the method addPlugInEventRepresentation
to enable a custom event representation, like this:
URI rootURI = new URI("type://mycompany/myproject/myname"); config.addPlugInEventRepresentation(rootURI, MyEventRepresentation.class.getName(), null);
The type://
part of the URI is an optional convention for the scheme part of an URI.
Esper allows invocations of static Java library functions in expressions, as outlined in Section 10.1, “Single-row Function Reference”. This configuration item can be set to allow a partial rather than a fully qualified class name in such invocations. The imports work in the same way as in Java files, so both packages and classes can be imported.
select Math.max(priceOne, PriceTwo) // via configuration equivalent to select java.lang.Math.max(priceOne, priceTwo)
Esper auto-imports the following Java library packages. Any additional imports that are specified in configuration files or through the API are added to the configuration in addition to the imports below.
java.lang.*
java.math.*
java.text.*
java.util.*
In a XML configuration file the auto-import configuration may look as below:
<auto-import import-name="com.mycompany.mypackage.*"/> <auto-import import-name="com.mycompany.myapp.MyUtilityClass"/>
Here is an example of providing imports via the API:
Configuration config = new Configuration(); config.addImport("com.mycompany.mypackage.*"); // package import config.addImport("com.mycompany.mypackage.MyLib"); // class import
In a XML configuration file the auto-import-annotation configuration may look as below:
<auto-import-annotation import-name="com.mycompany.mypackage.myannotations.*"/>
Here is an example of providing annotation-only imports via the API:
Configuration config = new Configuration(); // package import, only visible for annotation use config.addAnnotationImport("com.mycompany.mypackage.myannotations.*");
The LRU cache is described in detail in Section 16.4.10.6.1, “LRU Cache”. The expiry-time cache documentation can be found in Section 16.4.10.6.2, “Expiry-time Cache”
The next XML snippet is a sample cache configuration that applies to methods provided by the classes 'MyFromClauseLookupLib' and 'MyFromClauseWebServiceLib'. The XML and API configuration understand both the fully-qualified Java class name, as well as the simple class name:
<method-reference class-name="com.mycompany.MyFromClauseLookupLib"> <expiry-time-cache max-age-seconds="10" purge-interval-seconds="10" ref-type="weak"/> </method-reference> <method-reference class-name="MyFromClauseWebServiceLib"> <lru-cache size="1000"/> </method-reference>
<variable name="var_threshold" type="long" initialization-value="100"/> <variable name="var_key" type="string"/> <variable name="test" type="int" constant="true"/>
Please find the list of valid values for the type
attribute in Section 16.5, “Type Names”.
Esper has the capability to join event streams against historical data sources, such as a relational database. This section describes the configuration entries that the engine requires to access data stored in your database. Please see Section 5.13, “Accessing Relational Data via SQL” for information on the use of EPL queries that include historical data sources.
EPL queries that poll data from a relational database specify the name of the database as part of the EPL statement. The engine uses the configuration information described here to resolve the database name in the statement to database settings. The required and optional database settings are summarized below.
Database connections can be obtained via JDBC javax.xml.DataSource
, via java.sql.DriverManager
and via data source factory. Either one of these methods to obtain database connections is a required configuration.
Optionally, JDBC connection-level settings such as auto-commit, transaction isolation level, read-only and the catalog name can be defined.
Optionally, a connection lifecycle can be set to indicate to the engine whether the engine must retain connections or must obtain a new connection for each lookup and close the connection when the lookup is done (pooled).
Optionally, define a cache policy to allow the engine to retrieve data from a query cache, reducing the number of query executions.
Some of the settings can have important performance implications that need to be carefully considered in relationship to your database software, JDBC driver and runtime environment. This section attempts to outline such implications where appropriate.
The sample XML configuration file in the "etc" folder can be used as a template for configuring database settings. All settings are also available by means of the configuration API through the classes Configuration
and ConfigurationDBRef
.
<database-reference name="mydb1"> <datasource-connection context-lookup-name="java:comp/env/jdbc/mydb"> <env-property name="java.naming.factory.initial" value ="com.myclass.CtxFactory"/> <env-property name="java.naming.provider.url" value ="iiop://localhost:1050"/> </datasource-connection> </database-reference>
if (envProperties.size() > 0) { initialContext = new InitialContext(envProperties); } else { initialContext = new InitialContext(); } DataSource dataSource = (DataSource) initialContext.lookup(lookupName); Connection connection = dataSource.getConnection();
This section describes how to use Apache Commons Database Connection Pooling (Apache DBCP) with Esper. We also explain how to provide a custom application-specific DataSource
factory if not using Apache DBCP.
If your DataSource
provides connections out of a connection pool, your configuration should set the collection lifecycle setting to pooled
.
Apache DBCP provides comprehensive means to test for dead connections or grow and shrik a connection pool. Configuration properties for Apache DBCP can be found at Apache DBCP configuration. The listed properties are passed to Apache DBCP via the properties list provided as part of the Esper configuration.
The snippet of XML below is an example that configures a database named mydb3
to obtain connections via the pooling DataSource
provided by Apache DBCP BasicDataSourceFactory
.
The listed properties are passed to DBCP to instruct DBCP how to manage the connection pool. The settings below initialize the connection pool to 2 connections and provide the validation query select 1 from dual
for DBCP to validate a connection before providing a connection from the pool to Esper:
<database-reference name="mydb3"> <!-- For a complete list of properties see Apache DBCP. --> <!-- NOTE: "dbcp2" applies to api-2.0 of DBCP, use "dbcp" otherwise. --> <datasourcefactory-connection class-name="org.apache.commons.dbcp2.BasicDataSourceFactory"> <env-property name="username" value ="myusername"/> <env-property name="password" value ="mypassword"/> <env-property name="driverClassName" value ="com.mysql.jdbc.Driver"/> <env-property name="url" value ="jdbc:mysql://localhost/test"/> <env-property name="initialSize" value ="2"/> <env-property name="validationQuery" value ="select 1 from dual"/> </datasourcefactory-connection> <connection-lifecycle value="pooled"/> </database-reference>
The same configuration options provided through the API:
Properties props = new Properties(); props.put("username", "myusername"); props.put("password", "mypassword"); props.put("driverClassName", "com.mysql.jdbc.Driver"); props.put("url", "jdbc:mysql://localhost/test"); props.put("initialSize", 2); props.put("validationQuery", "select 1 from dual"); ConfigurationDBRef configDB = new ConfigurationDBRef(); // BasicDataSourceFactory is an Apache DBCP import configDB.setDataSourceFactory(props, BasicDataSourceFactory.class.getName()); configDB.setConnectionLifecycleEnum(ConfigurationDBRef.ConnectionLifecycleEnum.POOLED); Configuration configuration = new Configuration();; configuration.addDatabaseReference("mydb3", configDB);
Apache Commons DBCP is a separate download and not provided as part of the Esper distribution. The Apache Commons DBCP jar file requires the Apache Commons Pool jar file.
Your application can provide its own factory implementation for DataSource
instances: Set the class name to the name of the application class that provides a public static method named createDataSource
which takes a single Properties
object as parameter and returns a DataSource
implementation. For example:
configDB.setDataSourceFactory(props, MyOwnDataSourceFactory.class.getName()); ... class MyOwnDataSourceFactory { public static DataSource createDataSource(Properties properties) { return new MyDataSourceImpl(properties); } }
<database-reference name="mydb"> ... configure data source or driver manager settings... <expiry-time-cache max-age-seconds="60" purge-interval-seconds="120" /> </database-reference>
<database-reference name="mydb"> ... configure data source or driver manager settings... <expiry-time-cache max-age-seconds="60" purge-interval-seconds="120" ref-type="hard"/> </database-reference>
The next code snippet shows how to control this feature:
Configuration config = new Configuration(); config.getEngineDefaults().getThreading().setListenerDispatchPreserveOrder(false); engine = EPServiceProviderManager.getDefaultProvider(config);
And the XML configuration file can also control this feature by adding the following elements:
<engine-settings> <defaults> <threading> <listener-dispatch preserve-order="true" timeout-msec="1000" locking="spin"/> </threading> </defaults> </engine-settings>
The XML configuration file may change settings via the following elements:
<engine-settings> <defaults> <threading> <insert-into-dispatch preserve-order="true" timeout-msec="100" locking="spin"/> </threading> </defaults> </engine-settings>
The XML configuration file may change settings via the following elements:
<engine-settings> <defaults> <threading> <named-window-consumer-dispatch preserve-order="true" locking="spin"/> </threading> </defaults> </engine-settings>
The next code snippet shows how to disable the internal timer thread via the configuration API:
Configuration config = new Configuration(); config.getEngineDefaults().getThreading().setInternalTimerEnabled(false);
<engine-settings> <defaults> <threading> <internal-timer enabled="true" msec-resolution="200"/> </threading> </defaults> </engine-settings>
<engine-settings> <defaults> <threading> <threadpool-inbound enabled="true" num-threads="2"/> <threadpool-outbound enabled="true" num-threads="2" capacity="1000"/> <threadpool-timerexec enabled="true" num-threads="2"/> <threadpool-routeexec enabled="true" num-threads="2"/> </threading> </defaults> </engine-settings>
This example uses the API for configuring inbound threading :
Configuration config = new Configuration(); config.getEngineDefaults().getThreading().setThreadPoolInbound(true); config.getEngineDefaults().getThreading().setThreadPoolInboundNumThreads(2);
The XML configuration to enable fair locking, which is disabled by default, is as follows:
<engine-settings> <defaults> <threading engine-fairlock="true"/> </defaults> </engine-settings>
The API to change the setting:
Configuration config = new Configuration(); config.getEngineDefaults().getThreading().setEngineFairlock(true);
For example, consider the following query:
select propertyOne, propertyTwo from MyEvent
The XML snippet below is an example of setting the default event representation to Object-array:
<esper-configuration <engine-settings> <defaults> <event-meta> <event-representation type="objectarray"/> </event-meta> </defaults> </engine-settings> </esper-configuration>
configuration.getEngineDefaults().getEventMeta(). setDefaultEventRepresentation(Configuration.EventRepresentation.OBJECTARRAY);
To control the case sensitivity as discussed in Section 16.4.1.6, “Case Sensitivity and Property Names”, add the style
attribute in the XML configuration to set a default case sensitivity applicable to all event classes unless specifically overridden by class-specific configuration.
The default case sensitivity is case_sensitive
(case sensitivity turned on).
To control the accessor style as discussed in Section 16.4.1.3, “Non-JavaBean and Legacy Java Event Classes”, add the accessor-style
attribute in the XML configuration to set a default accessor style applicable to all event classes unless specifically overridden by class-specific configuration. The default accessor style is javabean
JavaBean accessor style.
The next code snippet shows how to control this feature via the API:
Configuration config = new Configuration(); config.getEngineDefaults().getEventMeta().setClassPropertyResolutionStyle( Configuration.PropertyResolutionStyle.CASE_INSENSITIVE); config.getEngineDefaults().getEventMeta().setDefaultAccessorStyle( ConfigurationEventTypeLegacy.AccessorStyle.PUBLIC);
The next code snippet outlines the API to turn off view resource sharing between statements:
Configuration config = new Configuration(); config.getEngineDefaults().getViewResources().setShareViews(false);
A code sample that turns iterable-unbound on is:
Configuration config = new Configuration(); config.getEngineDefaults().getViewResources().setIterableUnbound(true);
Here is a sample statement that specifies multiple expiry policies:
select * from MyEvent.std:unique(price).std:unique(quantity)
By default Esper applies intersection as described in Section 5.4.4, “Multiple Data Window Views”.
Here is the setting to allow multiple data windows without the intersection default:
Configuration config = new Configuration(); config.getEngineDefaults().getViewResources().setAllowMultipleExpiryPolicies(true);
When setting this option to true, and when using multiple data window views for a given stream, the behavior is as follows: The top-most data window receives an insert stream of events. It passes each insert stream event to each further data window view in the chain. Each data window view may remove events according to its expiry policy. Such remove stream events are only passed to data window views further in the chain, and are not made available to data window views earlier in the chain.
It is recommended to leave the default setting at false.
Statement-level processing information can be output via the @Audit
annotation, please see Section 17.3.1, “@Audit Annotation”.
When debug-level logging is enabled by setting the flag as below and by setting DEBUG in the Log4j configuration file, then the timer processing may produce extensive debug output that
you may not want to have in the log file. The timer-debug
setting in the XML or via API as below disables timer debug output which is enabled by default.
The API to use to enable debug logging and disable timer event output is shown here:
Configuration config = new Configuration(); config.getEngineDefaults().getLogging().setEnableExecutionDebug(true); config.getEngineDefaults().getLogging().setEnableTimerDebug(false);
Note: this is a configuration option that applies to all engine instances of a given Java module or VM.
The XML snippet is:
<esper-configuration> <engine-settings> <defaults> <logging> <execution-path enabled="true"/> <timer-debug enabled="false"/> </logging> </defaults> </engine-settings> </esper-configuration>
The API to use to enable query plan logging is shown here:
Configuration config = new Configuration(); config.getEngineDefaults().getLogging().setEnableQueryPlan(true);
<esper-configuration> <engine-settings> <defaults> <logging> <query-plan enabled="true"/> </logging> </defaults> </engine-settings> </esper-configuration>
The API to use to enable JDBC query logging is shown here:
Configuration config = new Configuration(); config.getEngineDefaults().getLogging().setEnableJDBC(true);
<esper-configuration> <engine-settings> <defaults> <logging> <jdbc enabled="true"/> </logging> </defaults> </engine-settings> </esper-configuration>
The settings herein control the output format of @Audit
logs.
The API to use to set am audit log format is shown here:
Configuration config = new Configuration(); config.getEngineDefaults().getLogging().setAuditPattern("[%u] [%s] %m");
The XML snippet is:
<esper-configuration> <engine-settings> <defaults> <logging> <audit pattern="[%u] [%s]%m"/> </logging> </defaults> </engine-settings> </esper-configuration>
For statements that use variables and that execute (in response to a single timer or other event) longer then the time period, the engine returns the current variable version at the time the statement executes, thereby softening the guarantee of consistency of variable values within the long-running statement. Please see Section 5.17.3, “Using Variables” for more information.
The XML configuration for this setting is shown below:
<engine-settings> <defaults> <variables> <msec-version-release value="15000"/> </variables> </defaults> </engine-settings>
The implications of the settings described herein are also detailed in Section 7.5.8.2, “Limiting Engine-wide Sub-Expression Count”.
A sample XML configuration for this setting is shown below:
<engine-settings> <defaults> <patterns> <max-subexpression value="100" prevent-start="false"/> </patterns> </defaults> </engine-settings>
The limit can be changed and disabled or enabled at runtime via the runtime configuration API. Pass a null value as the limit to disable limit checking.
A sample code snippet that sets a new limit is:
epService.getEPAdministrator().getConfiguration().setPatternMaxSubexpressions(100L);
The implications of the settings described herein are also detailed in Section 8.11, “Limiting Engine-wide State Count”.
A sample XML configuration for this setting is shown below:
<engine-settings> <defaults> <match-recognize> <max-state value="100" prevent-start="false"/> </match-recognize> </defaults> </engine-settings>
The limit can be changed and disabled or enabled at runtime via the runtime configuration API. Pass a null value as the limit to disable limit checking.
A sample code snippet that sets a new limit is:
epService.getEPAdministrator().getConfiguration().setMatchRecognizeMaxStates(100L);
A sample XML configuration for this setting is shown below:
<engine-settings> <defaults> <scripts default-dialect="js"/> </defaults> </engine-settings>
A sample code snippet that sets a new script dialect is:
config.getEngineDefaults().getScripts().setDefaultDialect("js");
The XML configuration for this setting is shown below:
<engine-settings> <defaults> <stream-selection> <stream-selector value="irstream" /> </stream-selection> </defaults> </engine-settings>
The equivalent code snippet using the configuration API is here:
Configuration config = new Configuration(); config.getEngineDefaults().getStreamSelection() .setDefaultStreamSelector(StreamSelector.RSTREAM_ISTREAM_BOTH);
By default, the internal timer uses the call System.currentTimeMillis()
to determine engine time in milliseconds. Via this setting the internal timer can be instructed to use System.nanoTime()
instead. Please see Section 15.9, “Time Resolution” for more information.
Note: This is a Java VM global setting. If running multiple engine instances in a Java VM, the timer setting is global and applies to all engine instances in the same Java VM, for performance reasons.
A sample XML configuration for this setting is shown below, whereas the sample setting sets the time source to the nanosecond time provider:
<engine-settings> <defaults> <time-source> <time-source-type value="nano" /> </time-source> </defaults> </engine-settings>
The equivalent code snippet using the configuration API is here:
Configuration config = new Configuration(); config.getEngineDefaults().getTimeSource(). setTimeSourceType(ConfigurationEngineDefaults.TimeSourceType.NANO);
A sample XML configuration is shown below:
<engine-settings> <defaults> <metrics-reporting jmx-engine-metrics="true"/> </defaults> </engine-settings>
A sample code snippet to set this configuration via the API follows:
configuration.getEngineDefaults().getMetricsReporting().setJmxEngineMetrics(true);
This section explains how to enable and configure metrics reporting, which is by default disabled. Please see Section 15.15, “Engine and Statement Metrics Reporting” for more information on the metrics data reported to your application.
The flag that enables metrics reporting is global to a Java virtual machine. If metrics reporting is enabled, the overhead incurred for reporting metrics is carried by all engine instances per Java VM.
Metrics reporting occurs by an engine-controlled separate daemon thread that each engine instance starts at engine initialization time, if metrics reporting and threading is enabled (threading enabled is the default).
Engine and statement metric intervals are in milliseconds. A negative or zero millisecond interval value may be provided to disable reporting.
To control statement metric reporting for individual statements or groups of statements, the engine provides a facility that groups statements by statement name. Each such statement group may have different reporting intervals configured, and intervals can be changed at runtime through runtime configuration. A statement group is assigned a group name at configuration time to identify the group.
Metrics reporting configuration is part of the engine default settings. All configuration options are also available via the Configuration
API.
A sample XML configuration is shown below:
<engine-settings> <defaults> <metrics-reporting enabled="true" engine-interval="1000" statement-interval="1000" threading="true"/> </defaults> </engine-settings>
The engine-interval
setting (defaults to 10 seconds) determines the frequency in milliseconds at which the engine reports engine metrics, in this example every 1 second. The statement-interval
is for statement metrics. The threading
flag is true by default since reporting takes place by a dedicated engine thread and can be set to false to use the external or internal timer thread instead.
The next example XML declares a statement group: The statements that have statement names that fall within the group follow a different reporting frequency:
<metrics-reporting enabled="true" statement-interval="0"> <stmtgroup name="MyStmtGroup" interval="2000" default-include="true" num-stmts="100" report-inactive="true"> <exclude-regex>.*test.*</exclude-regex> </stmtgroup> </metrics-reporting>
The above example configuration sets the statement-interval
to zero to disable reporting for all statements. It defines a statement group by name MyStmtGroup
and specifies a 2-second interval. The example sets the default-include
flag to true (by default false) to include all statements in the statement group. The example also sets report-inactive
to true (by default false) to report inactive statements.
The exclude-regex
element may be used to specify a regular expression that serves to exclude statements from the group. Any statement whose statement name matches the exclude regular expression is not included in the group. In the above example, all statements with the characters 'test' inside their statement name are excluded from the group.
Any statement not belonging to any of the statement groups follow the configured statement interval.
There are additional elements available to include and exclude statements: include-regex
, include-like
and exclude-like
. The latter two apply SQL-like matching. All patterns are case-sensitive.
Here is a further example of a possible statement group definition, which includes statements whose statement name have the characters @REPORT
or @STREAM
, and excludes statements whose statement name have the characters @IGNORE
or @METRICS
inside.
<metrics-reporting enabled="true"> <stmtgroup name="MyStmtGroup" interval="1000"> <include-like>%@REPORT%</include-like> <include-regex>.*@STREAM.*</include-like> <exclude-like>%@IGNORE%</exclude-like> <exclude-regex>.*@METRICS.*</exclude-regex> </stmtgroup> </metrics-reporting>
The XML configuration sets the locale dependent sorting as shown below:
<engine-settings> <defaults> <language sort-using-collator="true"/> </defaults> </engine-settings>
The API to change the setting:
Configuration config = new Configuration(); config.getEngineDefaults().getLanguage().setSortUsingCollator(true);
The XML configuration is as follows:
<engine-settings> <defaults> <expression integer-division="false" division-by-zero-is-null="false"/> </defaults> </engine-settings>
The API to change the setting:
Configuration config = new Configuration(); config.getEngineDefaults().getExpression().setIntegerDivision(true); config.getEngineDefaults().getExpression().setDivisionByZeroReturnsNull(true);
The XML configuration as below sets the same as the default value:
<engine-settings> <defaults> <expression self-subselect-preeval="true"/> </defaults> </engine-settings>
Here is a sample statement that utilitzes a sub-select against the same-events:
select * from MyEvent where prop not in (select prop from MyEvent.std:unique(otherProp))
The XML configuration as below sets the same as the default value:
<engine-settings> <defaults> <expression udf-cache="true"/> </defaults> </engine-settings>
The XML configuration as below sets the same as the default value:
<engine-settings> <defaults> <expression extend-agg="true"/> </defaults> </engine-settings>
The XML configuration as below sets the same as the default value:
<engine-settings> <defaults> <expression ducktyping="false"/> </defaults> </engine-settings>
The below XML configuration sets precision to 2 and rounding mode ceiling:
<engine-settings> <defaults> <expression math-context="precision=2 roundingMode=CEILING"/> </defaults> </engine-settings>
An example API configuration is shown next:
config.getEngineDefaults().getExpression().setMathContext(...);
By setting this configuration, the engine executes statements, when an event or schedule matches multiple statements, according to the assigned priority, starting from the highest priority value. See built-in EPL annotations in Section 5.2.7.7, “@Priority”.
By enabling this setting view sharing between statements as described in Section 16.4.13.1, “Sharing View Resources between Statements” is disabled.
The XML configuration to enable the flag, which is disabled by default, is as follows:
<engine-settings> <defaults> <execution prioritized="true"/> </defaults> </engine-settings>
The API to change the setting:
Configuration config = new Configuration(); config.getEngineDefaults().getExecution().setPrioritized(true);
The XML configuration to enable fair locking, which is disabled by default, is as follows:
<engine-settings> <defaults> <execution fairlock="true"/> </defaults> </engine-settings>
The API to change the setting:
Configuration config = new Configuration(); config.getEngineDefaults().getExecution().setFairlock(true);
The XML configuration to disable context level locking is as follows:
<engine-settings> <defaults> <execution disable-locking="true"/> </defaults> </engine-settings>
The API to change the setting:
Configuration config = new Configuration(); config.getEngineDefaults().getExecution().setDisableLocking(true);
The XML configuration to set a large threading profile is as follows:
<engine-settings> <defaults> <execution threading-profile="large"/> </defaults> </engine-settings>
The API to change the setting:
Configuration config = new Configuration(); config.getEngineDefaults().getExecution().setThreadingProfile( ConfigurationEngineDefaults.ThreadingProfile.LARGE);
The XML configuration to set a new filter service profile is as follows:
<engine-settings> <defaults> <execution filter-server-profile="readwrite"/> </defaults> </engine-settings>
The API to change the setting:
Configuration config = new Configuration(); config.getEngineDefaults().getExecution(). setFilterServiceProfile(ConfigurationEngineDefaults.FilterServiceProfile.READWRITE);
This setting is for performance tuning of filter expression analysis and breakdown.
On the level of an EPL statement, this setting can be controlled by providing a hint. For example:
// The engine optimizes the filter expression to become: // "a=1, c=1" or "b=1, c=1" or "a=1, d=1" or "b=1, d=1". // This enables reverse index sharing between filter expressions. select * from Event((a=1 or b=1) and (c=1 or d=1))
// The engine does not optimize filter expressions @Hint('MAX_FILTER_WIDTH=0') select * from Event((a=1 or b=1) and (c=1 or d=1))
The XML configuration to sets a new engine-wide value:
<engine-settings> <defaults> <execution filter-service-max-filter-width="100"/> </defaults> </engine-settings>
The API to change the setting:
Configuration config = new Configuration(); config.getEngineDefaults().getExecution(). setFilterServiceMaxFilterWidth(16);
The XML configuration to enable isolated service providers is as follows:
<engine-settings> <defaults> <execution allow-isolated-service="true"/> </defaults> </engine-settings>
The API to change the setting:
Configuration config = new Configuration(); config.getEngineDefaults().getExecution().setAllowIsolatedService(true);
Use the settings as described here to register an exception handler factory class that provides an exception handler. The engine invokes exception handlers in the order they are listed to handle a continues-query unchecked exception, as further described in Section 15.11, “Exception Handling”.
Please provide the full-qualified class name of each class that implements the com.espertech.esper.client.hook.ExceptionHandlerFactory
interface in the engine defaults configuration as below.
By default, during a module undeploy when the engine encounters a runtime exception for any of the statements it logs such exceptions as warnings.
You can set the undeploy-rethrow-policy
flag to rethrow_first
instead have the engine rethrow the first runtime exception.
The XML configuration is as follows:
<engine-settings> <defaults> <exceptionHandling undeploy-rethrow-policy="warn"> <handlerFactory class="my.company.cep.MyCEPEngineExceptionHandlerFactory"/> </exceptionHandling> </defaults> </engine-settings>
The API calls to register an exception handler factory are as follows:
Configuration config = new Configuration(); config.getEngineDefaults().getExceptionHandling().addClass(MyCEPEngineExceptionHandlerFactory.class); config.getEngineDefaults().getExceptionHandling().setUndeployRethrowPolicy(ConfigurationEngineDefaults.ExceptionHandling.UndeployRethrowPolicy.RETHROW_FIRST);
Use the settings as described here to register a condition handler factory class that provides a condition handler. The engine invokes condition handlers in the order they are listed to indicate conditions, which is the term used for notification when certain predefined limits are reached, as further described in Section 15.12, “Condition Handling”.
Please provide the full-qualified class name of each class that implements the com.espertech.esper.client.hook.ConditionHandlerFactory
interface in the engine defaults configuration as below.
The XML configuration is as follows:
<engine-settings> <defaults> <conditionHandling> <handlerFactory class="my.company.cep.MyCEPEngineConditionHandlerFactory"/> </conditionHandling> </defaults> </engine-settings>
The API calls to register a condition handler factory are as follows:
Configuration config = new Configuration(); config.getEngineDefaults().getConditionHandling().addClass(MyCEPEngineConditionHandlerFactory.class);
Revision event types reflect a versioning relationship between events of same or different event types. Please refer to Section 2.10, “Updating, Merging and Versioning Events” and Section 6.11, “Versioning and Revision Event Type Use with Named Windows”.
The configuration consists of the following:
An name of an event type whose events are base events.
Zero, one or more names of event types whose events are delta events.
One or more property names that supply the key values that tie base and delta events to existing revision events. Properties must exist on the event type as simple properties. Nested, indexed or mapped properties are not allowed.
Optionally, a strategy for overlaying or merging properties. The default strategy is Overlay Declared as described below.
The XML configuration for this setting is shown below:
<revision-event-type name="UserProfileRevisions"> <base-event-type name="ProfileCreation"/> <delta-event-type name="ProfileUpdate"/> <key-property name="userid"/> </revision-event-type>
If configuring via runtime or initialization-time API, this code snippet explains how:
Configuration config = new Configuration(); ConfigurationRevisionEventType configRev = new ConfigurationRevisionEventType(); configRev.setNameBaseEventType("ProfileCreation"); configRev.addNameDeltaEventType("ProfileUpdate"); configRev.setKeyPropertyNames(new String[] {"userid"}); config.addRevisionEventType("UserProfileRevisions", configRev);
As the configuration provides names of base and delta event types, such names must be configured for JavaBean, Map or XML events as the previous sections outline.
The next table outlines the available strategies:
<variant-stream name="OrderStream"> <variant-event-type name="PartsOrder"/> <variant-event-type name="ServiceOrder"/> </variant-stream>
This code snippet sets up a variant stream by name OutgoingEvent
:
Configuration config = new Configuration(); ConfigurationVariantStream variant = new ConfigurationVariantStream(); variant.setTypeVariance(ConfigurationVariantStream.TypeVariance.ANY); config.addVariantStream("OutgoingEvent", variant);
Esper logs all messages to Apache commons logging under an appropriate log level. To output log messages you can add Log4j to classpath and configure Log4j as below.
Esper's only direct dependency for logging is the Apache Commons Logging interfaces. You may use Log4j as described here, or you may use SLF4J instead (for example) as described in http://www.slf4j.org/legacy.html. You can also redirect SLF4J to any backend you want - nop, logback, jul as needed.
Statement-level processing information can be output, please see Section 17.3.1, “@Audit Annotation”.
For performance reasons, Esper does not log any debug-level or informational-level messages for event execution unless explicitly configured via Section 16.4.14.1, “Execution Path Debug Logging”.
A callback API for receiving certain critical engine reports is available as described in Section 15.11, “Exception Handling”.
More information on configuring engine-level settings for logging are at Section 16.4.14, “Engine Settings related to Logging”.
Apache Commons Logging uses an automatic discovery mechanism to find a log framework that it will delegate to. If your application has another component using logback, please inspect you logback configuration and add com.espertech
.
The next table explains the log levels:
Log4j is the default logging component. Please find additional information for Log4j configuration and extension in http://logging.apache.org/log4j.
The easiest way to configure Log4j is by providing a Log4J configuration file, similar to the log4j.xml
file shipped in the etc
folder of the distribution.
Add the log4j.configuration
system property to the java
command line and provide the file name of the Log4j configuration file, making sure your classpath also includes the directory of the file:
java -Dlog4j.configuration=log4j.xml ...