A Jdbc based implementation of Lucene Directory allowing the storage of a Lucene index within a database. Enables existing or new Lucene based application to store the Lucene index in a database with no or minimal change to typical Lucene code fragments.

The JdbcDirectory is highly configurable, using the optional JdbcDirectorySettings. All the settings are described in the javadoc, and most of them will be made clear during the next sections.

There are several options to instantiate a Jdbc directory, they are:

The Jdbc directory works against a single table (where the table name must be provided when the directory is created). The table schema is described in the following table:

The Jdbc directory provides the following operations on top of the ones forced by the Directory interface:

The Jdbc directory requires a Dialect implementation that is specific to the database used with it. The following is a table listing the current dialects supported with the Jdbc directory:

* A Blob locator is a pointer to the actual data, which allows fetching only portions of the Blob at a time. Databases (or Jdbc drivers) that do not use locators usually fetch all the Blob data for each query (which makes using them impractical for large indexes). Note, the support documented here does not cover all the possible Jdbc drivers, please refer to your Jdbc driver documentation for more information.

Minor performance improvements can be gained if JdbcTable is cached and used to create different JdbcDirectory instances.

It is best to use a pooled data source (like Jakarta Commons DBCP), so Connections won't get created every time, but be pooled.

Most of the time, when working with Jdbc directory, it is best to work in a non compound index format. Since with databases there is no problem of too many files open, it won't be an issue. The package comes with a set of utilities to compound or uncompund an index, located in the org.apache.lucene.index.LuceneUtils class, just in case you already have an index and it is in the wrong structure.

When indexing data, a possible performance improvement can be to index the data into the file system or memory, and then copy over the contents of the index to the database. org.apache.lucene.index.LuceneUtils comes with a utility to copy one directory to the other, and changing the compound state of the index while copying.

JdbcDirectory performs no transaction management. All database related operations WITHIN IT work in the following manner:

Connection conn = DataSourceUtils.getConnection(dataSource);
// perform any database related operation using the connection
DataSourceUtils.releaseConnection(conn);

As you can see, no commit or rollback are called on the connection, allowing for any type of transaction management done outside of the actual JdbcDirectory related operations. Also, the fact that we are using the Jdbc DataSource, allows for plug able transaction management support (usually based on DataSource delegate and Connection proxy). DataSourceUtils is a utility class that comes with the Jdbc directory, and it's usage will be made clear in the following sections.

There are several options when it comes to transaction management, and they are:

JdbcDirectory jdbcDir = // ... create the jdbc directory
Connection conn = DataSourceUtils.getConnection(dataSource);
try {
    IndexReader indexReader = new IndexReader(jdbcDir); // you can also use an already open IndexReader
    // ...
    DataSourceUtils.commitConnectionIfPossible(conn); // will commit the connection if controlling it
} catch (IOException e) {
    DataSourceUtils.safeRollbackConnectionIfPossible(conn);
    throw e;
} finnaly {
    DataSourceUtils.releaseConnection(conn);
}

DirectoryTemplate template = new DirectoryTemplate(dir); // use a pre-configured directory
template.execute(new DirectoryTemplate.DirectoryCallbackWithoutResult() {
    protected void doInDirectoryWithoutResult(Directory dir) throws IOException {
        IndexWriter writer = new IndexWriter(dir, new SimpleAnalyzer(), true);
        // index write operations
        write.close();
    }
});

// or, for example, if we have a cached IndexSearcher

template.execute(new DirectoryTemplate.DirectoryCallbackWithoutResult() {
    protected void doInDirectoryWithoutResult(Directory dir) throws IOException {
        // indexSearcher operations
    }
});

A FileEntryHandler is an interface used by the Jdbc directory to delegate file level operations to it. The JdbcDirectorySettings has a default file entry handler which handles all unmapped file names. It also provides the ability to register a FileEntryHandler against either an exact file name, or a file extension (3 characters after the '.').

When the JdbcDirectory is created, all the different file entry handlers that are registered with the directory settings are created and configured. They will than be used to handle files based on the file names.

When registering a new file entry handler, it must be registered with JdbcFileEntrySettings. The JdbcFileEntrySettings is a fancy wrapper around java Properties in order to provide an open way for configuring file entry handlers. When creating a new JdbcFileEntrySettings it already has sensible defaults (refer to the javadoc for them), but of course they can be changed. One important configuration setting is the type of the FileEntryHandler, which should be set under the constant setting name: JdbcFileEntrySettings#FILE_ENTRY_HANDLER_TYPE and should be the fully qualified class name of the file entry handler.

The Jdbc directory package comes with three different FileEntryHandlers. They are:

Most of the files use the MarkDeleteFileEntryHandler, since there might be other currently open IndexReaders or IndexSearchers that use the files. The JdbcDirectory provide the deleteMarkDeleted() and deleteMarkDeleted(delta) to actually purge old entries that are marked as deleted. It should be scheduled and executed once in a while in order to keep the database table compact.

When creating new JdbcDirectorySettings, it already registers different file entry handlers for specific files automatically. For example, the deleted file is registered against a NoOpFileEntryHandler since we will always be able to delete entries from the database (the deleted file is used to store files that could not be deleted from the file system). This results in better performance since no operations are executed against the deleted (or deleted related files). Another example, is registering the ActualDeleteFileEntryHandler against the segments file, since we do want to delete it and replace it with a new one when it is written.