The Compass Needle Terracotta integration allows to store a Lucene index in a distributed manner using Terracotta as well as provide seamless integration with Compass.

Compass provides a Terracotta optimized directory (similar to Lucene RAM directory) called TerracottaDirectory. When using it with pure Lucene applications, the directory needs to be defined as a "root" Terracotta object and then used when constructing IndexWriter and IndexReader. See the Compass Store on how to use compass jar file as a Terracotta Integration Module (TIM).

Terracotta is a shared memory (referred to as "network attached memory"). The terracotta directory makes use of that and stores the directory in memory allowing for terracotta to distribute changes of it to all relevant nodes connected to the terracotta server. The actual content of a "file" in the directory is broken down into one or more byte arrays, which can be controlled using the bufferSize parameter. Note, once an index is created with a certain bufferSize, it should not be changed. By default, the buffer size is set to 4096 bytes.

Terracotta will automatically fetch required content from the server, and will evict content if memory thresholds break for an application. When constructing large files, the directory allows to set a flush rate when the file content will be flushed (and be allowed to be evicted) during its creation. The formula is that every bufferSize * flushRate bytes, it will be released by Compass and allow terracotta to move it to the server and reclaim the memory. The default flush rate is set to 10.

When using Compass, it is very simple to configure Compass to store the index using Terracotta. Compass jar file already comes in the format of a Terracotta Integration Module (TIM) allowing to simply drop it into TC_HOME/modules and it already comes pre-configured with a terracotta configuration of both locks and roots (terracotta.xml file within the root of the compass jar file). Another option is to tell Terracotta where to look for more TIMs within the application tc-config file and point it to where the compass jar is located.

Once the TIM is setup, Compass has a special Terracotta connection that allows it to use the TerracottaDirectory called TerracottaDirectoryStore. The TerracottaDirectoryStore is where terracotta is configured to have its root (note, this is all defined for you already since compass is a TIM).

Here is a properties/settings based configuration

compass.engine.connection=tc://myindex
# default values, just showing how it can be configured
compass.engine.store.tc.bufferSize=4096  
compass.engine.store.tc.flushRate=10

And here is an xml based configuration:

<compass name="default">
  <connection>
      <tc indexName="myindex" bufferSize="4096" flushRate="10" />
  </connection>
</compass>

The "client application" will need to run using Terracotta bootclasspath configuration, and have the following in its tc-config.xml:

  <clients>
      <modules>
          <module group-id="org.compass-project" name="compass" version="2.0.0-RC1" />
      </modules>
  </clients>

For more information on how to run it in different ways/environments, please refer to the terracotta documentation.