Chapter 2. GETTING STARTED WITH THE NDB API

Table of Contents

2.1. Compiling and Linking NDB API Programs
2.1.1. General Requirements
2.1.2. Compiler Options
2.1.3. Linker Options
2.1.4. Using Autotools
2.2. Connecting to the Cluster
2.2.1. Include Files
2.2.2. API Initialisation and Cleanup
2.2.3. Establishing the Connection
2.3. Mapping MySQL Database Object Names and Types to NDB

Abstract

This chapter discusses preparations for writing an NDB API application.

2.1. Compiling and Linking NDB API Programs

Abstract

This section provides information on compiling and linking NDB API applications, including requirements and compiler and linker options.

2.1.1. General Requirements

To use the NDB API with MySQL, you must have the NDB client library and its header files installed alongside the regular MySQL client libraries and headers. These are automatically installed when you build MySQL using the --with-ndbcluster configure option or when using a MySQL binary package that supports the NDBCluster storage engine.

Note

MySQL 4.1 does not install the required NDB-specific header files. You should use MySQL 5.0 or later when writing NDB API applications, and this Guide is targeted for use with MySQL 5.1.

2.1.2. Compiler Options

Header Files.  In order to compile source files that use the NDB API, you must ensure that the necessary header files can be found. Header files specific to the NDB API are installed in the following subdirectories of the MySQL include directory:

  • include/mysql/storage/ndb/ndbapi

  • include/mysql/storage/ndb/mgmapi

Compiler Flags.  The MySQL-specific compiler flags needed can be determined using the mysql_config utility that is part of the MySQL installation:

$ mysql_config --cflags
-I/usr/local/mysql/include/mysql -Wreturn-type -Wtrigraphs -W -Wformat
-Wsign-compare -Wunused  -mcpu=pentium4 -march=pentium4

This sets the include path for the MySQL header files but not for those specific to the NDB API. The --include option to mysql_config returns the generic include path switch:

shell> mysql_config --include
-I/usr/local/mysql/include/mysql

It is necessary to add the subdirectory paths explicitly, so that adding all the needed compile flags to the CXXFLAGS shell variable should look something like this:

CFLAGS="$CFLAGS "`mysql_config --cflags`
CFLAGS="$CFLAGS "`mysql_config --include`storage/ndb
CFLAGS="$CFLAGS "`mysql_config --include`storage/ndb/ndbapi
CFLAGS="$CFLAGS "`mysql_config --include`storage/ndb/mgmapi

Tip

If you do not intend to use the Cluster management functions, the last line in the previous example can be omitted. However, if you are interested in the management functions only, and do not want or need to access Cluster data except from MySQL, then you can omit the line referencing the ndbapi directory.

2.1.3. Linker Options

NDB API applications must be linked against both the MySQL and NDB client libraries. The NDB client library also requires some functions from the mystrings library, so this must be linked in as well.

The necessary linker flags for the MySQL client library are returned by mysql_config --libs. For multithreaded applications you should use the --libs_r instead:

$ mysql_config --libs_r
-L/usr/local/mysql-5.1/lib/mysql -lmysqlclient_r -lz -lpthread -lcrypt 
-lnsl -lm -lpthread -L/usr/lib -lssl -lcrypto

To link an NDB API application, it is necessary to add -lndbclient and -lmystrings to these options. Adding all the required linker flags to the LDFLAGS variable should look something like this:

LDFLAGS="$LDFLAGS "`mysql_config --libs_r`
LDFLAGS="$LDFLAGS -lndbclient -lmystrings"

2.1.4. Using Autotools

It is often faster and simpler to use GNU autotools than to write your own makefiles. In this section, we provide an autoconf macro WITH_MYSQL that can be used to add a --with-mysql option to a configure file, and that automatically sets the correct compiler and linker flags for given MySQL installation.

All of the examples in this chapter include a common mysql.m4 file defining WITH_MYSQL. A typical complete example consists of the actual source file and the following helper files:

  • acinclude

  • configure.in

  • Makefile.m4

automake also requires that you provide README, NEWS, AUTHORS, and ChangeLog files; however, these can be left empty.

To create all necessary build files, run the following:

aclocal
autoconf
automake -a -c
configure --with-mysql=/mysql/prefix/path

Normally, this needs to be done only once, after which make will accommodate any file changes.

Example 1-1: acinclude.m4

m4_include([../mysql.m4])

Example 1-2: configure.in

AC_INIT(example, 1.0)
AM_INIT_AUTOMAKE(example, 1.0)
WITH_MYSQL()
AC_OUTPUT(Makefile)

Example 1-3: Makefile.am

bin_PROGRAMS = example
example_SOURCES = example.cc

Example 1-4: WITH_MYSQL source for inclusion in acinclude.m4

dnl
dnl configure.in helper macros
dnl 
 
AC_DEFUN([WITH_MYSQL], [ 
  AC_MSG_CHECKING(for mysql_config executable)

  AC_ARG_WITH(mysql, [  --with-mysql=PATH path to mysql_config binary or mysql prefix dir], [
  if test -x $withval -a -f $withval
    then
      MYSQL_CONFIG=$withval
    elif test -x $withval/bin/mysql_config -a -f $withval/bin/mysql_config
    then 
     MYSQL_CONFIG=$withval/bin/mysql_config
    fi
  ], [
  if test -x /usr/local/mysql/bin/mysql_config -a -f /usr/local/mysql/bin/mysql_config
    then
      MYSQL_CONFIG=/usr/local/mysql/bin/mysql_config
    elif test -x /usr/bin/mysql_config -a -f /usr/bin/mysql_config
    then
      MYSQL_CONFIG=/usr/bin/mysql_config
    fi
  ])

  if test "x$MYSQL_CONFIG" = "x" 
  then
    AC_MSG_RESULT(not found)
    exit 3
  else
    AC_PROG_CC
    AC_PROG_CXX
 
    # add regular MySQL C flags
    ADDFLAGS=`$MYSQL_CONFIG --cflags` 

    # add NDB API specific C flags
    IBASE=`$MYSQL_CONFIG --include`
    ADDFLAGS="$ADDFLAGS $IBASE/ndb"
    ADDFLAGS="$ADDFLAGS $IBASE/ndb/ndbapi"
    ADDFLAGS="$ADDFLAGS $IBASE/ndb/mgmapi"

    CFLAGS="$CFLAGS $ADDFLAGS"    
    CXXFLAGS="$CXXFLAGS $ADDFLAGS"    

    LDFLAGS="$LDFLAGS "`$MYSQL_CONFIG --libs_r`" -lndbclient -lmystrings"
    
    AC_MSG_RESULT($MYSQL_CONFIG)
  fi  
])

2.2. Connecting to the Cluster

Abstract

This section covers connecting an NDB API application to a MySQL cluster.

2.2.1. Include Files

NDB API applications require one or more of the following include files:

  • Applications accessing Cluster data via the NDB API must include the file NdbApi.hpp.

  • Applications making use of both the NDB API and the regular MySQL client API also need to include mysql.h.

  • Applications that use cluster management functions need the include file mgmapi.h.

2.2.2. API Initialisation and Cleanup

Before using the NDB API, it must first be initialised by calling the ndb_init() function. Once an NDB API application is complete, call ndb_end(0) to perform a cleanup.

2.2.3. Establishing the Connection

To establish a connection to the server, it is necessary to create an instance of Ndb_cluster_connection, whose constructor takes as its argument a cluster connectstring; if no connectstring is given, localhost is assumed.

The cluster connection is not actually initiated until the Ndb_cluster_connection::connect() method is called. When invoked without any arguments, the connection attempt is retried each 1 second indefinitely until successful, and no reporting is done. See Section 3.2, “The Ndb_cluster_connection Class”, for details.

By default an API node will connect to the “nearest” data node — usually a data node running on the same machine, due to the fact that shared memory transport can be used instead of the slower TCP/IP. This may lead to poor load distribution in some cases, so it is possible to enforce a round-robin node connection scheme by calling the set_optimized_node_selection() method with 0 as its argument prior to calling connect(). (See Section 3.2.1.4, “Ndb_cluster_connection::set_optimized_node_selection().)

The connect() method initiates a connection to a cluster management node only — it does not wait for any connections to data nodes to be made. This can be accomplished by using wait_until_ready() after calling connect(). The wait_until_ready() method waits up to a given number of seconds for a connection to a data node to be established.

In the following example, initialisation and connection are handled in the two functions example_init() and example_end(), which will be included in subsequent examples via the file example_connection.h.

Example 2-1: Connection example. 

#include <stdio.h>
#include <stdlib.h>

#include <NdbApi.hpp>

#define CONNECTSTR "localhost"

Ndb_cluster_connection* example_init()
{
  Ndb_cluster_connection* conn;

  // initialise MySQL and Ndb client libraries
  if( ndb_init() )
  {
    exit(EXIT_FAILURE);
  }

  // prepare connection to cluster
  conn = new Ndb_cluster_connection(CONNECTSTR);

  // initiate connection
  if( conn->connect(4, 5, 1) )
  {
    fprintf(stderr, "Unable to connect to cluster within 30 seconds.");
    exit(EXIT_FAILURE);
  }
  
  // wait for data (ndbd) nodes
  if(conn->wait_until_ready(30, 0) < 0)
  {
    fprintf(stderr, "Cluster was not ready within 30 seconds.\n");
    exit(EXIT_FAILURE);
  }

  return conn; 
}

void example_end(Ndb_cluster_connection* conn)
{
  // terminate connection
  delete conn;

  // shut down MySQL and Ndb client libraries
  ndb_end(2);
}


int main(int argc, char** argv)
{
  Ndb_cluster_connection* conn;

  conn = example_connect();

  printf("Connection established.");

  example_end(conn);

  return EXIT_SUCCESS;
}

2.3. Mapping MySQL Database Object Names and Types to NDB

Abstract

This section discusses NDB naming and other conventions with regard to database objects.

Databases and Schemas.  Databases and schemas are not represented by objects as such in the NDB API. Instead, they are modelled as attributes of Table and Index objects. The value of the database attribute of one of these objects is always the same as the name of the MySQL database to which the table or index belongs. The value of the schema attribute of a Table or Index object is always 'def' (for “default”).

Tables.  MySQL table names are directly mapped to NDB table names without modification. Table names starting with 'NDB$' are reserved for internal use>, as is the SYSTAB_0 table in the sys database.

Indexes.  There are two different type of NDB indexes:

  • Hash indexes are unique, but not ordered.

  • B-tree indexes are ordered, but allow duplicate values.

Names of unique indexes and primary keys are handled as follows:

  • For a MySQL UNIQUE index, both a B-tree and a hash index are created. The B-tree index uses the MySQL name for the index; the name for the hash index is generated by appending '$unique' to the index name.

  • For a MySQL primary key only a B-tree index is created. This index is given the name PRIMARY. There is no extra hash; however, the uniqueness of the primary key is guaranteed by making the MySQL key the internal primary key of the NDB table.

Column Names and Values.  NDB column names are the same as their MySQL names.

Datatypes.  MySQL datatypes are stored in NDB columns as follows:

  • The MySQL TINYINT, SMALLINT, INT, and BIGINT datatypes map to NDB types having the same names and storage requirements as their MySQL counterparts.

  • The MySQL FLOAT and DOUBLE datatypes are mapped to NDB types having the same names and storage requirements.

  • The storage space required for a MySQL CHAR column is determined by the maximum number of characters and the column's character set. For most (but not all) character sets, each character takes one byte of storage. When using UTF-8, each character requires three bytes. You can find the number of bytes needed per character in a given character set by checking the Maxlen column in the output of SHOW CHARACTER SET.

  • In MySQL 5.1, the storage requirements for a VARCHAR or VARBINARY column depend on whether the column is stored in memory or on disk:

    • For in-memory columns, the NDBCLUSTER storage engine supports variable-width columns with 4-byte alignment. This means that (for example) a the string 'abcde' stored in a VARCHAR(50) column using the latin1 character set requires 12 bytes — in this case, 2 bytes times 5 characters is 10, rounded up to the next even multiple of 4 yields 12. (This represents a change in behaviour from Cluster in MySQL 5.0 and 4.1, where a column having the same definition required 52 bytes storage per row regardless of the length of the string being stored in the row.)

    • In Disk Data columns, VARCHAR and VARBINARY are stored as fixed-width columns. This means that each of these types requires the same amount of storage as a CHAR of the same size.

  • Each row in a Cluster BLOB or TEXT column is made up of two separate parts. One of these is of fixed size (256 bytes), and is actually stored in the original table. The other consists of any data in excess of 256 bytes, which stored in a hidden table. The rows in this second table are always 2000 bytes long. This means that record of size bytes in a TEXT or BLOB column requires

    • 256 bytes, if size <= 256

    • 256 + 2000 * ((size – 256) \ 2000) + 1) bytes otherwise