The Compass Needle Coherence integration allows to store a Lucene index within Coherence Data Grid.

Compass provides two types of Lucene Directory implementations using Coherence, DataGridCoherenceDirectory and InvocableCoherenceDirectory.

Here is a simple example of how it can be used:

CoherenceDirectory dir = new InvocableCoherenceDirectory("cacheName", "indexName");
// ... (use the dir with IndexWriter and IndexSearcher)
dir.close();

In the above example we created the invocable Coherence directory on top of Coherence's Data Grid with an index named "test". The directory can now be used to create Lucene IndexWriter and IndexSearcher.

The Lucene directory interface represents a virtual file system. Implementing it on top of Coherence is done by breaking files into a file header, called FileEntryKey/FileEntryValue and one or more FileBucketKey/FileBucketValue. The file header holds the meta data of the file, for example, its size and timestamp, while the file bucket holds a bucket size of the actual file content. The bucket size can be controlled when constructing the coherence directory, but note that it must not be changed if connecting to an existing index.

The DataGridCoherenceDirectory uses coherence features that are supported by all of coherence editions. It uses coherence lock API and plain Map remove APIs. The InvocableCoherenceDirectory uses coherence invocation service support allowing to delete files (header and buckets) in parallel (without returning results), and use FileLockKey existence as an indicator for locking (conditional put) which results in better performance (for remove operations) and better lock API implementation.

Note, it is preferable to configure the directory not to use the compound index format as it yields better performance (note, by default, when using Compass, the non compound format will be used). Also, the merge factor for the directory (also applies to Compass optimizers) should be set to a higher value (than the default 10) since it mainly applies to file based optimizations.

The Coherence integration can also use Coherence just as a distributed lock manager without the need to actually store the index on Coherence. Either the InvocableCoherenceLockFactory or DefaultCoherenceLockFactory can be used for it.

Compass allows for simple integration with DataGridCoherenceDirectory and InvocableCoherenceDirectory as the index storage mechanism. The following example shows how Compass can be configured to work against an invocable coherence directory based index with an index named test and cache name named testcache:

<compass name="default">
  <connection>
      <coherence indexName="test" cacheName="testcache"/>
  </connection>
</compass>

The following shows how to configure it using properties based configuration:

compass.engine.connection=coherence://test:testcache

By default, when using Coherence as the Compass store, the index will be in an uncompound file format. It will also automatically be configured with an expiration time based index deletion policy so multiple clients will work correctly.

Compass can also be configured just to used Coherence as a distributed lock manager without the need to actually store the index on Coherence (note that when configuring Cohernece as the actual store, the Coherence lock factory will be used by default). Here is how it can be configured:

compass.engine.store.lockFactory.type=org.compass.needle.cohernece.InvocableCoherenceLockFactoryProvider
compass.engine.store.lockFactory.path=cacheName

Compass comes with a simple CacheStore implementation called CompassCacheStore. The implementation implements only the cache store API (cache load returns null) and relies on the fact that the key of the hash map is the key of the object (note, Compass supports composite ids using the id-component mapping). The implementation allows to simple write behind of changes happening in the cache to be reflected in the index (which, of course, can be stored in coherence as well).