Specification of QDBM Version 1

Copyright (C) 2000-2003 Mikio Hirabayashi
Last Update: Thu, 20 Feb 2003 21:45:58 +0900

Table of Contents

  1. Overview
  2. Features
  3. Installation
  4. Depot: Basic API
  5. Commands for Depot
  6. Curia: Extended API
  7. Commands for Curia
  8. Relic: Compatible API
  9. Commands for Relic
  10. Bugs
  11. Copying

(I'm not good at English. So please proofread this document and point out mistakes to me.)


Overview

QDBM is a library of routines for managing a database. The database is a simple data file containing records, each is a pair of a key and a value. Every key and value is serial bytes with variable length. Both binary data and character string can be used as a key and a value. There is neither concept of data tables nor data types. Each key must be unique within a database, so it is impossible to store two or more records with a key overlaps.

The following access methods are provided to the database: storing a record with a key and a value, deleting a record by a key, retrieving a record by a key. Moreover, traversal access to every key are provided, but the order is arbitrary. These access methods are similar to ones of DBM (or its compatibles, NDBM and GDBM) library defined in the UNIX standard. QDBM is an alternative for DBM because of its higher performance.


Features

QDBM is developed referring to GDBM for the purpose of the following three points: higher processing speed, smaller size of a database file, and simpler API. The features of QDBM are just having attained them. Moreover, the following three restrictions of DBM: a process can handle only one database, the size of a key and a value is bounded, a database file is sparse, are cleared.

QDBM uses hash algorithm to retrieve records. If a bucket array has sufficient number of elements, the time complexity of retrieval is `O(1)'. That is, time required for retrieving a record is constant, regardless of the scale of the database. It is also the same about storing and deleting. Collision of hash values is managed by separate chain method. Data structure of the chain is binary search tree. Even if a bucket array has unusually scarce elements, the time complexity of retrieval is `O(log n)'.

QDBM attains improvement in retrieval by loading whole of a bucket array onto RAM. If a bucket array is on RAM, it is possible to access a region of a target record by about one pass of file operations. A bucket array saved in a file is not read into RAM with the `read' call but directly mapped to RAM with the `mmap' call. Therefore, preparation time on connecting to a database is very short, and two or more processes can share the same memory map.

If number of elements of a bucket array is about half of records stored within a database, although it depends on characteristic of the input, the rate of collision of hash values are about 56.7% (36.8% if the same, 21.3% if twice, 11.5% if four times, 6.0% if eight times). In such case, it is possible to retrieve a record by two or less passes of file operations. If it is made into a performance index, in order to handle a database containing one million of records, a bucket array with a half million of elements is needed. The size of each element is 4 bytes. That is, if 2M bytes of RAM is available, a database containing one million records can be handled.

QDBM has two kinds of methods to connect to a database. A `reader' can perform retrieving but neither storing nor deleting. A `writer' can perform all access methods. Exclusion control between processes is performed when connecting to a database by file locking. While a writer is connecting to a database, neither readers nor writers can connect. While a reader is connecting to a database, other readers can connect, but neither writers can. According to this mechanism, the adjustment of simultaneous connections to a database is guaranteed in multitasking environment. However, it does not correspond in multithread environment.

DBM has two modes of storing operation, `insert' and `replace'. In case a key overlaps an existing record, the insert mode keeps the existing value, while the replace mode transposes it to the specified value. In addition to the two modes, QDBM has `concatenate' mode. In the mode, the specified value is concatenated at the end of the existing value and stored. This feature is useful when adding a element to a value as an array. Moreover, although DBM has a method to fetch out a value from a database only by reading a whole region of a record, QDBM has a method to fetch out a part of a region of a value. When a value is treated as an array, this feature is also useful.

If data alignment is assigned to a database, each record will place in a file with vacating suitable padding bytes. When it is going to overwrite a value of larger size than the size of the existing value or to concatenate a specified value to a existing value, if increasing size is settled in size of the padding, it is not necessary to move the region of the existing record to another position. Although the processing which moves the region of a record requires amount of calculation according to size of the region, the performance of updating database to the scale of stored records is kept constant by taking padding in size according to size of each record.

QDBM has three kinds of APIs: the basic API, the extended API and the compatible API. In the basic API, a database is treated as a file. In the extended API, a database is treated as a directory containing one or more database files. If it is going to store all the data of a database in one file, file size may exceed restriction of a file system. In the extended API, a database is divided into two or more files in a directory. Since the basic API and the extended API are resembled mutually, it is easy to porting an application between each API. The compatible API is for porting an application from NDBM to QDBM. Besides, QDBM has command line interfaces corresponding to the three APIs. They are useful for unit test, debugging applications and so on. They will also support prototyping of database applications with script languages.


Installation

To install QDBM, `gcc' and `make' are required.

When an archive file of QDBM is extracted, change the current working directory to the generated directory and perform installation.

Run the configuration script.

./configure

Build programs.

make

Perform self-diagnostic test.

make check

Install programs. This operation must be carried out by the root user.

make install

When a series of work finishes, header files, `depot.h', `curia.h' and `relic.h' will be installed in `/usr/local/include', libraries, `libqdbm.a' and `libqdbm.so' will be installed in `/usr/local/lib', executable commands, `dpmgr', `dptest', `dptsv', `crmgr', `crtest', `rlmgr' and `rltest' will be installed in `/usr/local/bin'. Besides, on Windows (Cygwin), libraries and its destinations of installation are differ from ones of generic UNIX environments. `libqdbm.a' and `libqdbm.dll.a' will be installed in `/usr/local/lib'. `qdbm.dll' will be installed in `$SYSTEMROOT/system32'.

To uninstall QDBM, execute the following command after `./configure'. This operation must be carried out by the root user.

make uninstall

If an old version of QDBM is installed on your system, uninstall it before installation of a new version.


Depot: Basic API

Depot is the basic API of QDBM. In order to use Depot, you should include `depot.h' and `stdlib.h' in source code. Usually, the following description will be near the beginning of a source file.

#include <depot.h>
#include <stdlib.h>

A pointer to DEPOT is used as a database handle. It is like that some file I/O routines of `stdio.h' use a pointer to FILE. A database handle is opened with the function `dpopen' and closed with `dpclose'. You should not refer directly to any member of a handle. If a fatal error occurs in a database, any access method via the handle except `dpclose' will not work and return error status.

You can assign a file descriptor for debugging information to the external variable `dpdbgfd'. The initial value is -1. If the value is positive, some debugging information is output into the file descriptor.

extern int dpdbgfd;

A status of the last happened error is assigned to the external variable `dpecode'. The initial value is is DP_ENOERR. Refer to `depot.h' for details of the type DPECODE.

extern DPECODE dpecode;

You can use the function `dperrmsg' in order to obtain a message string corresponding to an error code.

const char *dperrmsg(DPECODE ecode);
`ecode' specifies an error code. The return value is a message string of an error message. Region of the return value is not writable.

You can use the function `dpopen' in order to obtain a database handle. Size of a bucket array of a database is determined on creating, and is not to be changed except for by optimization of the database. Suggested size of a bucket array is about from 0.5 to 4 times of number of all records to store. While connecting as a writer, an exclusive lock is invoked to the database. While connecting as a reader, a shared lock is invoked to the database. Control blocks until the lock is achieved.

DEPOT *dpopen(const char *name, DPOMODE omode, int bnum);
`name' specifies a name of a database file. `omode' specifies a connection mode: DP_OWRITER as a writer, DP_OREADER as a reader. If the mode is DP_OWRITER, the following may be added by bitwise or: DP_OCREAT, which means it creates a new database if not exist, DP_OTRUNC, which means it creates a new database regardless if one exists. `bnum' specifies number of elements of a bucket array. If it is not more than 0, the default value is specified. The return value is a database handle. If the return value is NULL, it is not successful.

Every connected handle is to be closed with the function `dpclose'. Contents of updating of a database is synchronized with a file when closing connection. When a writer opens a database, a database will be destroyed if it does not close appropriately.

int dpclose(DEPOT *depot);
`depot' specifies a database handle. If successful, the return value is true, else, it is false. Since the region of the closed handle is released, it becomes impossible to use the handle.

You can use the function `dpput' in order to store a recored.

int dpput(DEPOT *depot, const char *kbuf, int ksiz, const char *vbuf, int vsiz, DPDMODE dmode);
`depot' specifies a database handle connected as a writer. `kbuf' specifies a pointer to the bytes of a key. `ksiz' specifies size of the region of a key. If `ksiz' is negative, the size assigned with `strlen(kbuf)'. `vbuf' specifies a pointer to the bytes of a value. `vsiz' specifies size of the region of the value. If `vsiz' is negative, the size assigned with `strlen(vbuf)'. `dmode' specifies behavior when the key overlaps, by the following values: DP_DOVER, which means a specified value overwrites an existing one, DP_DKEEP, which means an existing value is kept, DP_DCAT, which means a the specified value is concatenated at the end of the existing value and stored. If successful, the return value is true, else, it is false.

You can use the function `dpout' in order to delete a recored.

int dpout(DEPOT *depot, const char *kbuf, int ksiz);
`depot' specifies a database handle connected as a writer. `kbuf' specifies a pointer to the bytes of a key. `ksiz' specifies size of the region of a key. If `ksiz' is negative, the size assigned with `strlen(kbuf)'. If successful, the return value is true, else, it is false. false is returned when no record corresponds to the specified key.

You can use the function `dpget' in order to retrieve a recored. The region of the return value is allocated with the `malloc' call, so is to be released with the `free' call.

char *dpget(DEPOT *depot, const char *kbuf, int ksiz, int start, int max, int *sp);
`depot' specifies a database handle. `kbuf' specifies a pointer to the bytes of a key. `ksiz' specifies size of the region of a key. If `ksiz' is negative, the size assigned with `strlen(kbuf)'. `start' specifies an offset address of the beginning of a value's region to be read. `max' specifies max length of bytes to be read. if `max' is negative, length to read is unlimited. `sp' specifies a pointer to a variable to which length of the region of the return value. If `sp' is NULL, it is not used. If successful, the return value is a pointer to the region of the value of the corresponding record, else, it is NULL. false is returned when no record corresponds to the specified key or length of the value of the record is less than `start'. Because an additional nil code is append at the end of the region of the return value, the return value can be treated as a character string.

You can use the function `dpvsiz' in order to know length of a value of a recored. The function is faster than `dpget'.

int dpvsiz(DEPOT *depot, const char *kbuf, int ksiz);
`depot' specifies a database handle. `kbuf' specifies a pointer to the bytes of a key. `ksiz' specifies size of the region of a key. If `ksiz' is negative, the size assigned with `strlen(kbuf)'. If successful, the return value is length of the value of the corresponding record, else, it is -1. -1 is returned when no record corresponds to the specified key.

You can use the function `dpiterinit' in order to initialize a iterator of a database handle for traversal access to every key in the database.

int dpiterinit(DEPOT *depot);
`depot' specifies a database handle. If successful, the return value is true, else, it is false.

You can use the function `dpiternext' in order to get the next key of a iterator. Although the order cannot be controlled, every record in a database can be referred to. The region of the return value is allocated with the `malloc' call, so is to be released with the `free' call.

char *dpiternext(DEPOT *depot, int *sp);
`depot' specifies a database handle. `sp' specifies a pointer to a variable to which length of the region of the return value. If `sp' is NULL, it is not used. If successful, the return value is a pointer to the region of the next key, else, it is NULL. NULL is returned when no record is to be get out of the iterator. Because an additional nil code is append at the end of the region of the return value, the return value can be treated as a character string.

You can use the function `dpsetalign' in order to set alignment of a database handle. If alignment is assigned in a database handle, processing speed of overwriting an existing record goes up. Basic size of alignment is suggested to be average length of values of records to be stored. Unit size of alignment is suggested to be about from 4 times to 16 times of basic size.

int dpsetalign(DEPOT *depot, int asiz, int aunit);
`depot' specifies a database handle connected as a writer. `asiz' specifies basic size of alignment. `aunit' specifies unit size of alignment. If successful, the return value is true, else, it is false. When a record is recording, size of the region reserved for its value is determined as multiple of alignment size. Alignment size is determined as multiple of basic size by size of specified value divided by unit size. Since an alignment size is not saved in a database, you should specify alignment with opening a database.

You can use the function `dpsync' in order to synchronize contents of updating with a file and device. The function is useful when another process uses a connected database file.

int dpsync(DEPOT *depot);
`depot' specifies a database handle connected as a writer. If successful, the return value is true, else, it is false.

You can use the function `dpoptimize' in order to optimize a database. In an alternating succession of storing in overwrite mode or concatenation mode, size of a database file comes big due to accumulated dispensable spaces. Such dispensable spaces are deleted by optimization.

int dpoptimize(DEPOT *depot, int bnum);
`depot' specifies a database handle connected as a writer. `bnum' specifies number of elements of a bucket array. If it is not more than 0, the default value is specified.

You can use the function `dpname' in order to know a name of a database handle. The region of the return value is allocated with the `malloc' call, so is to be released with the `free' call.

char *dpname(DEPOT *depot);
`depot' specifies a database handle. If successful, the return value is a pointer to the region of the name of the database handle, else, it is NULL.

You can use the function `dpname' in order to know size of a database file.

int dpfsiz(DEPOT *depot);
`depot' specifies a database handle. If successful, the return value is size of the database file, else, it is -1.

You can use the function `dpbnum' in order to know number of elements which a bucket array of a database has.

int dpbnum(DEPOT *depot);
`depot' specifies a database handle. If successful, the return value is number of elements which the bucket array of the database has, else, it is -1.

You can use the function `dpbnum' in order to know number of used elements of a bucket array.

int dpbusenum(DEPOT *depot);
`depot' specifies a database handle. If successful, the return value is number of used elements of the bucket array, else, it is -1.

You can use the function `dpbnum' in order to know number of records stored in a database.

int dprnum(DEPOT *depot);
`depot' specifies a database handle. If successful, the return value is number of records stored in the database, else, it is -1.

You can use the function `dpwritable' in order to know whether a database handle is a writer or not.

int dpwritable(DEPOT *depot);
`depot' specifies a database handle. The return value is true if the handle is a writer, false if not.

You can use the function `dpfdesc' in order to get a file descriptor of a database file. However, handling a database file directly is not suggested.

int dpfdesc(DEPOT *depot);
`depot' specifies a database handle. The return value is a file descriptor of a database file.

The function `dpinnerhash' is a hash function used inside of Depot. The function is useful to know which element a key is to be stored into.

int dpinnerhash(const char *kbuf, int ksiz);
`kbuf' specifies a pointer to the bytes of a key. `ksiz' specifies size of the region of a key. If `ksiz' is negative, the size assigned with `strlen(kbuf)'. The return value is a hash value of 31 bits length computed from the key.

The function `dpouterhash' is a hash function without reference to hash functions used inside of Depot. The function is useful for hash algorithm used in an application of Depot.

int dpouterhash(const char *kbuf, int ksiz);
`kbuf' specifies a pointer to the bytes of a key. `ksiz' specifies size of the region of a key. If `ksiz' is negative, the size assigned with `strlen(kbuf)'. The return value is a hash value of 31 bits length computed from the key.

The function `dpprimenum' is used in order to obtain a prime number not less than a number. The function is useful when an application determine size of a bucket array of its hash algorithm.

int dpprimenum(int num);
`num' specified a positive number. The return value is a prime number not less than the specified number.

For building a program using Depot, the program should be linked with a library file `libqdbm.a' or `libqdbm.so'. For example, the following command is executed to build `hoge' from `hoge.c'.

gcc -I/usr/local/include -L/usr/local/lib -o hoge hoge.c -lqdbm

The following example stores and retrieves a phone number, using the name as the key.

#include <depot.h>
#include <stdlib.h>
#include <stdio.h>

#define NAME     "mikio"
#define NUMBER   "000-1234-5678"
#define DBNAME   "book"

/* function prototypes */
int main(int argc, char **argv);

/* main routine */
int main(int argc, char **argv){
  DEPOT *depot;
  char *val;

  /* open the database */
  if(!(depot = dpopen(DBNAME, DP_OWRITER | DP_OCREAT, -1))){
    fprintf(stderr, "%s\n", dperrmsg(dpecode));
    exit(1);
  }

  /* store the record */
  if(!dpput(depot, NAME, -1, NUMBER, -1, DP_DOVER)){
    fprintf(stderr, "%s\n", dperrmsg(dpecode));
  }

  /* retrieve the record */
  if(!(val = dpget(depot, NAME, -1, 0, -1, NULL))){
    fprintf(stderr, "%s\n", dperrmsg(dpecode));
  } else {
    printf("Name: %s\n", NAME);
    printf("Number: %s\n", val);
    free(val);
  }

  /* close the database */
  if(!dpclose(depot)){
    fprintf(stderr, "%s\n", dperrmsg(dpecode));
    exit(1);
  }

  return 0;
}

Commands for Depot

Depot has the following command line interfaces.

`dpmgr' is a utility for debugging Depot and its applications. It features editing and checking of a database. It can be use for database applications with shell script. `dpmgr' is used in the following format. `name' specifies a database name. `key' specifies a key of a record. `val' specifies a value of a record.

Create a database file.
dpmgr create [-v] [-bnum num] name
Storing a record with a key and a value.
dpmgr put [-v] [-kx] [-vx] [-keep] [-cat] name key val
Delete a record with a key.
dpmgr out [-v] [-kx] name key
Retrieving a record with a key and output to the standard output.
dpmgr get [-v] [-kx] [-start num] [-max num] [-ox] [-n] name key
List all keys delimited with linefeed to the standard output.
dpmgr list [-v] [-ox] name
Optimize a database.
dpmgr optimize [-v] [-bnum num] name
Output miscellaneous information to the standard output.
dpmgr inform [-v] name
Output version information of QDBM to the standard output.
dpmgr version
Options features the following.
-v : output debug information.
-bnum num : specifies number of elements of a bucket array.
-kx : treat `key' as a binary expression of hexadecimal notation.
-vx : treat `val' as a binary expression of hexadecimal notation.
-keep : specify storing mode for DP_OKEEP.
-cat : specify storing mode for DP_OCAT.
-start : specify beginning offset of a value to fetch.
-max : specify max length of a value to fetch.
-ox : treat the output as a binary expression of hexadecimal notation.
-n : do not output the trailing newline.

`dpmgr' returns 0 on success, another on failure.

`dptest' is a utility for facility test and performance test. Check a database generated by the command or measure the execution time of the command. `dptest' is used in the following format. `name' specifies a database name. `rnum' specifies a number of records. `bnum' specifies a number of elements of a bucket array.

Store records with keys of 8 bytes. It changing as `00000001', `00000002'...
dptest write [-cat num] [-asiz num] [-aunit aunit] name rnum bnum
Retrieve all records of the database above.
dptest read name
Perform combination test of variou operations.
dptest combo name
Store records with randam data with randam length.
dptest rand [-cat num] [-asiz num] [-aunit aunit] name rnum bnum
Options features the following.
-cat num : specify a number of repeating times and storing mode for DP_OCAT.
-asiz num : specify basic size of alignment.
-aunit num : specify unit size of alignment.

`dptest' returns 0 on success, another on failure.

`dptsv' features mutual coversion between database of Depot and TSV text. `dptsv' is used in the following format. `name' specifies a database name. `export' reads TSV data from the standard input. If a key overlaps, the latter is adopted. `-bnum' specifies a number of elements of a bucket array. `import' writes TSV data to the standard output.

Create a database from TSV.
dptsv import [-bnum num] name
Write TSV data of a database.
dptsv export name

`dptsv' returns 0 on success, another on failure.


Curia: Extended API

Curia is the extended API of QDBM. Curia provides routines for managing two or more database files in a directory. Restriction of some file systems that size of each file is limited is escaped by dividing a database file into two or more. Moreover, deployment of each database file on individual devices improves scalability.

In order to use Curia, you should include `depot.h', `curia.h' and `stdlib.h' in source code. Usually, the following description will be near the beginning of a source file.

#include <depot.h>
#include <curia.h>
#include <stdlib.h>

A pointer to CURIA is used as a database handle. A database handle is opened with the function `cropen' and closed with `crclose'. You should not refer directly to any member of a handle.

You can use the function `cropen' in order to obtain a database handle. Size of a bucket array of a database is determined on creating, and is not to be changed except for by optimization of the database. Suggested size of a bucket array is about from 0.5 to 4 times of number of all records to store. Max number of division of databases is 256. While connecting as a writer, an exclusive lock is invoked to the database. While connecting as a reader, a shared lock is invoked to the database. Control blocks until the lock is achieved.

CURIA *cropen(const char *name, CROMODE omode, int bnum, int dnum);
`name' specifies a name of a database file. `omode' specifies a connection mode: CR_OWRITER as a writer, CR_OREADER as a reader. If the mode is CR_OWRITER, the following may be added by bitwise or: CR_OCREAT, which means it creates a new database if not exist, CR_OTRUNC, which means it creates a new database regardless if one exists. `bnum' specifies number of elements of a bucket array. If it is not more than 0, the default value is specified. `dnum' specifies number of division of databases. If it is not more than 0, the default value is specified. The return value is a database handle. If the return value is NULL, it is not successful.

Every connected handle is to be closed with the function `crclose'. Contents of updating of a database is synchronized with a file when closing connection. When a writer opens a database, a database will be destroyed if it does not close appropriately.

int crclose(CURIA *curia);
`curia' specifies a database handle. If successful, the return value is true, else, it is false. Since the region of the closed handle is released, it becomes impossible to use the handle.

The following routines are to be used as well as ones of Depot. Besides, there is no function corresponding to `dperrmsg', `dpinnerhash', `dpouterhash' and `dpprimenum'.

int crput(CURIA *curia, const char *kbuf, int ksiz, const char *vbuf, int vsiz, CRDMODE dmode);
int crout(CURIA *curia, const char *kbuf, int ksiz);
char *crget(CURIA *curia, const char *kbuf, int ksiz, int start, int max, int *sp);
int crvsiz(CURIA *curia, const char *kbuf, int ksiz);
int criterinit(CURIA *curia);
char *criternext(CURIA *curia, int *sp);
int crsetalign(CURIA *curia, int asiz, int aunit);
int crsync(CURIA *curia);
int croptimize(CURIA *curia, int bnum);
char *crname(CURIA *curia);
int crfsiz(CURIA *curia);
int crbnum(CURIA *curia);
int crbusenum(CURIA *curia);
int crrnum(CURIA *curia);

Curia has the following functions for managing large objects. While usual records are saved in a few database files, large objects are saved in defferent files for each other. Because files of large objects are deployed in directories by hash value, speed of retrieving is not so slow.

You can use the function `crputlob' in order to store a large object.

int crputlob(CURIA *curia, const char *kbuf, int ksiz, const char *vbuf, int vsiz, CRDMODE dmode);
`curia' specifies a database handle connected as a writer. `kbuf' specifies a pointer to the bytes of a key. `ksiz' specifies size of the region of a key. If `ksiz' is negative, the size assigned with `strlen(kbuf)'. `vbuf' specifies a pointer to the bytes of a value. `vsiz' specifies size of the region of the value. If `vsiz' is negative, the size assigned with `strlen(vbuf)'. `dmode' specifies behavior when the key overlaps, by the following values: CR_DOVER, which means a specified value overwrites an existing one, CR_DKEEP, which means an existing value is kept, CR_DCAT, which means a the specified value is concatenated at the end of the existing value and stored. If successful, the return value is true, else, it is false.

You can use the function `croutlob' in order to delete a large object.

int croutlob(CURIA *curia, const char *kbuf, int ksiz);
`curia' specifies a database handle connected as a writer. `kbuf' specifies a pointer to the bytes of a key. `ksiz' specifies size of the region of a key. If `ksiz' is negative, the size assigned with `strlen(kbuf)'. If successful, the return value is true, else, it is false. false is returned when no record corresponds to the specified key.

You can use the function `crgetlob' in order to retrieve a large object. The region of the return value is allocated with the `malloc' call, so is to be released with the `free' call.

char *crgetlob(CURIA *curia, const char *kbuf, int ksiz, int start, int max, int *sp);
`curia' specifies a database handle. `kbuf' specifies a pointer to the bytes of a key. `ksiz' specifies size of the region of a key. If `ksiz' is negative, the size assigned with `strlen(kbuf)'. `start' specifies an offset address of the beginning of a value's region to be read. `max' specifies max length of bytes to be read. if `max' is negative, length to read is unlimited. `sp' specifies a pointer to a variable to which length of the region of the return value. If `sp' is NULL, it is not used. If successful, the return value is a pointer to the region of the value of the corresponding large object, else, it is NULL. false is returned when no large object corresponds to the specified key or length of the value of the large object is less than `start'. Because an additional nil code is append at the end of the region of the return value, the return value can be treated as a character string.
int crvsizlob(CURIA *curia, const char *kbuf, int ksiz);
`curia' specifies a database handle. `kbuf' specifies a pointer to the bytes of a key. `ksiz' specifies size of the region of a key. If `ksiz' is negative, the size assigned with `strlen(kbuf)'. If successful, the return value is length of the value of the corresponding large object, else, it is -1. -1 is returned when no large object corresponds to the specified key.
int crrnumlob(CURIA *curia);
`curia' specifies a database handle. If successful, the return value is number of large objects stored in the database, else, it is -1.

How to build programs using Curia is the same as the case of Depot.


Commands for Curia

Curia has the following command line interfaces.

`crmgr' is a utility for debugging Depot and its applications. It features editing and checking of a database. It can be use for database applications with shell script. `crmgr' is used in the following format. `name' specifies a database name. `key' specifies a key of a record. `val' specifies a value of a record.

Create a database file.
crmgr create [-v] [-bnum num] name
Storing a record with a key and a value.
crmgr put [-v] [-kx] [-vx] [-keep] [-cat] [-lob] name key val
Delete a record with a key.
crmgr out [-v] [-kx] [-lob] name key
Retrieving a record with a key and output to the standard output.
crmgr get [-v] [-kx] [-start num] [-max num] [-ox] [-lob] [-n] name key
List all keys delimited with linefeed to the standard output.
crmgr list [-v] [-ox] name
Optimize a database.
crmgr optimize [-v] [-bnum num] name
Output miscellaneous information to the standard output.
crmgr inform [-v] name
Output version information of QDBM to the standard output.
crmgr version
Options features the following.
-v : output debug information.
-bnum num : specifies number of elements of a bucket array.
-kx : treat `key' as a binary expression of hexadecimal notation.
-vx : treat `val' as a binary expression of hexadecimal notation.
-keep : specify storing mode for CR_OKEEP.
-cat : specify storing mode for CR_OCAT.
-start : specify beginning offset of a value to fetch.
-max : specify max length of a value to fetch.
-ox : treat the output as a binary expression of hexadecimal notation.
-lob : handle large objects.
-n : do not output the trailing newline.

`crmgr' returns 0 on success, another on failure.

`crtest' is a utility for facility test and performance test. Check a database generated by the command or measure the execution time of the command. `crtest' is used in the following format. `name' specifies a database name. `rnum' specifies a number of records. `bnum' specifies a number of elements of a bucket array.

Store records with keys of 8 bytes. It changing as `00000001', `00000002'...
crtest write [-cat num] [-asiz num] [-aunit aunit] name rnum bnum
Retrieve all records of the database above.
crtest read name
Perform combination test of variou operations.
crtest combo name
Options features the following.
-cat num : specify a number of repeating times and storing mode for DP_OCAT.
-asiz num : specify basic size of alignment.
-aunit num : specify unit size of alignment.

`crtest' returns 0 on success, another on failure.


Relic: Compatible API

Relic is the compatible API with NDBM. So, Relic wraps Depot as API of NDBM. It is easy to port an application from NDBM to QDBM. In most cases, you should only replace includings of `ndbm.h' with `relic.h'.

The original NDBM treats a database as a pair of two files. One has a name with suffix `.dir' and stores a bit map of keys. The other has a name with suffix `.pag' and stores entities of all data. Relic creates a file of `.dir' as a mere dummy file and creates a file of `.pag' as a database. Relic has no restriction about length of data of each record. Relic cannot handle database files made by the original NDBM.

In order to use Relic, you should include `relic.h', `sys/types.h', `sys/stat.h' and `fcntl.h' in source code. Usually, the following description will be near the beginning of a source file.

#include <relic.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

A pointer to DBM is used as a database handle. A database handle is opened with the function `dbm_open' and closed with `dbm_close'. You should not refer directly to any member of a handle.

Structures of `datum' type is used in order to give and receive data of keys and values with functions of Relic.

typedef struct { void *dptr; size_t dsize; } datum;
`dptr' specifies a pointer to the bytes of a key. `dsize' specifies size of the region of a key.

You can use the function `dbm_open' in order to obtain a database handle.

DBM *dbm_open(char *name, int flags, int mode);
`name' specifies a name of a database. Filenames are concatenated with suffixes. `flags' is the same as one of `open' call, although O_WRONLY is treated as O_RDWR and additional flags except for O_CREAT and O_TRUNC have no effect. `mode' specifies mode of database files as one of `open' call does. The return value is a database handle. If the return value is NULL, it is not successful.

Every connected handle is to be closed with the function `dbm_close'.

void dbm_close(DBM *db);
`db' specifies a database handle. Since the region of the closed handle is released, it becomes impossible to use the handle.

You can use the function `dbm_store' in order to store a recored.

int dbm_store(DBM *db, datum key, datum content, int flags);
`db' specifies a database handle. `key' specifies a structure of a key. `content' specifies a structure of a value. `flags' specifies behavior when the key overlaps, by the following values: DBM_REPLACE, which means a specified value overwrites an existing one, DBM_INSERT, which means an existing value is kept. The return value is 0 if it is successful, 1 if it gives up because of overlaps of the key, -1 if other error occurs.

You can use the function `dbm_delete' in order to delete a recored.

int dbm_delete(DBM *db, datum key);
`db' specifies a database handle. `key' specifies a structure of a key. The return value is 0 if it is successful, -1 if some errors occur.

You can use the function `dbm_fetch' in order to retrieve a recored.

datum dbm_fetch(DBM *db, datum key);
`db' specifies a database handle. `key' specifies a structure of a key. The return value is a structure of a value. If a record corresponds, a member `dptr' of the structure is a pointer to the region of the record. If no record corresponds or some errors occur, `dptr' is NULL. `dptr' points to a region related with the handle. The region is available until the next time of calling the following functions: `dbm_fetch', `dbm_firstkey' and `dbm_nextkey' with the same handle. Because an additional nil code is append at the end of the region of the return value, the return value can be treated as a character string.

You can use the function `dbm_firstkey' in order to get the first key of a database. However, order of records is arbitrary.

datum dbm_firstkey(DBM *db);
`db' specifies a database handle. The return value is a structure of a value. If a record corresponds, a member `dptr' of the structure is a pointer to the region of the record. If no record corresponds or some errors occur, dptr is NULL. `dptr' points to a region related with the handle. The region is available until the next time of calling the following functions: `dbm_fetch', `dbm_firstkey' and `dbm_nextkey' with the same handle. Because an additional nil code is append at the end of the region of the return value, the return value can be treated as a character string.

You can use the function `dbm_nextkey' in order to get the next key of a database. However, order of records is arbitrary.

datum dbm_nextkey(DBM *db);
`db' specifies a database handle. The return value is a structure of a value. If a record corresponds, a member `dptr' of the structure is a pointer to the region of the record. If no record corresponds or some errors occur, dptr is NULL. `dptr' points to a region related with the handle. The region is available until the next time of calling the following functions: `dbm_fetch', `dbm_firstkey' and `dbm_nextkey' with the same handle. Because an additional nil code is append at the end of the region of the return value, the return value can be treated as a character string.

The function `dbm_error' has no effect.

int dbm_error(DBM *db);
`db' specifies a database handle. The return value is 0. The function is only for compatibility.

The function `dbm_clearerr' has no effect.

int dbm_clearerr(DBM *db);
`db' specifies a database handle. The return value is 0. The function is only for compatibility.

You can use the function `dbm_rdonly' in order to check whether a handle is read-only or not.

int dbm_rdonly(DBM *db);
`db' specifies a database handle. The return value is true if the handle is read-only, or false if not read-only.

You can use the function `dbm_dirfno' in order to get a file descriptor of a directory file. However, handling a database file directly is not suggested.

int dbm_dirfno(DBM *db);
`db' specifies a database handle. The return value is a file descriptor of a directory file.

You can use the function `dbm_pagfno' in order to get a file descriptor of a data file. However, handling a database file directly is not suggested.

int dbm_pagfno(DBM *db);
`db' specifies a database handle. The return value is a file descriptor of a data file.

How to build programs using Relic is the same as the case of Depot. Note that an option to be given to a linker is not `-lndbm', but `-lqdbm'.


Commands for Relic

Relic has the following command line interfaces.

`rlmgr' is a utility for debugging Relic and its applications. It features editing and checking of a database. It can be use for database applications with shell script. `rlmgr' is used in the following format. `name' specifies a database name. `key' specifies a key of a record. `val' specifies a value of a record.

Create a database file.
rlmgr create name
Storing a record with a key and a value.
rlmgr store [-kx] [-vx] [-insert] name key val
Delete a record with a key.
rlmgr delete [-kx] name key
Retrieving a record with a key and output to the standard output.
rlmgr fetch [-kx] [-ox] [-n] name key
List all keys delimited with linefeed to the standard output.
rlmgr list [-v] [-ox] name
Options features the following.
-kx : treat `key' as a binary expression of hexadecimal notation.
-vx : treat `val' as a binary expression of hexadecimal notation.
-insert : specify storing mode for DBM_INSERT.
-ox : treat the output as a binary expression of hexadecimal notation.
-n : do not output the trailing newline.

`rlmgr' returns 0 on success, another on failure.

`rltest' is a utility for facility test and performance test. Check a database generated by the command or measure the execution time of the command. `rltest' is used in the following format. `name' specifies a database name. `rnum' specifies a number of records.

Store records with keys of 8 bytes. It changing as `00000001', `00000002'...
rltest write name rnum
Retrieve records of the database above.
rltest read name rnum

`rltest' returns 0 on success, another on failure.


Bugs

There is no bug which are found but not fixed. If you find any bug, report it to the author.

Database files are vulnerable for I/O error. QDBM has neither feature for rollback nor backup. Applications are responsible for handling such errors as disk-full and killing interruptive signals.


Copying

This program is written by Mikio Hirabayashi and distributed as a free software. You can redistribute it and/or modify it under the terms of the GNU General Public License Version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

You may contact the author by e-mail to <mikio@24h.co.jp>. Any suggestion or bug report is welcome to the author.