Data types in MMBase

Authors: André van Toly, Michiel Meeuwissen
Date: 2007-08-03

This software is OSI Certified Open Source Software. OSI Certified is a certification mark of the Open Source Initiative.

The license (Mozilla version 1.0) can be read at the MMBase site. See http://www.mmbase.org/license

1. MMBase Data Types

MMBase datatypes are defined in the resource <MMBase configuration>/datatypes.xml. See the Xml Schema Definition, for a formal description of this XML format. This resource can exist multiple times, but there is at least one in mmbase.jar. The contents of that one are described in Section 7, “Summary of default datatypes”

Other jars (e.g. of MMBase components) can also provide such a datatypes.xml, which will also be read in, and added to the repository of available datatypes.

Of course you can also place your own datatypes.xml, e.g. just in WEB-INF/config.

On a running MMBase you can see which datatypes are available in /mmbase/validation/. See e.g. at /mmbase/validation at the mmbase.org site.

2. Description of the Data-type XML

A datatypes.xml basically contains a list of 'datatype' entries. E.g.

<?xml version="1.0" ?>
<datatypes
    xmlns="http://www.mmbase.org/xmlns/datatypes"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.mmbase.org/xmlns/datatypes http://www.mmbase.org/xmlns/datatypes.xsd">


  <datatype id="integer">
    <description xml:lang="en">
      Not so very big integer numbers. Common base for all kind of enumerations.
    </description>
    <class name="org.mmbase.datatypes.IntegerDataType" />
  </datatype>
  ...

There are only a limited number of data types which can be defined like 'integer' in this example. Namely only the database types of MMBase (string, integer, binary, float, double, long, xml, node, datetime and boolean). All other datatypes are based on one of those. This can be done in two ways. As a sub element 'specialization':

<datatype id="float">
  <specialization id="scale1-10">
    <description xml:lang="en">A scale from 1 to 10.</description>
    <minInclusive value="1.0"  />
    <maxInclusive value="10.0"  />
  </specialization>
</datatype>

or as a 'datatype' element with a 'base' attribute.

<datatype base="float" id="scale1-10">
  <description xml:lang="en">A scale from 1 to 10.</description>
  <minInclusive value="1.0"  />
  <maxInclusive value="10.0"  />
</datatype>

It is also possible to replace a data type, by extending from the data type with the same id. This is only feasible in WEB-INF/config/datatypes because that one is parsed last.

<!-- In this site, email addresses are required -->
<datatype base="emailaddress" id="emailaddress">
  <required value="true">
    <description xml:lang="nl">Je moet een emailadres intypen</description>
    <description xml:lang="fr">Un address courriel est requis</description>
    <description xml:lang="en">An email address is required</description>
  </required>
</datatype>

In the following subsections we will elaborate a bit on the possible sub-elements of a datatype (or 'specialization') element.

2.1. Description, class, default value, and properties

You can start with descriptions (one for each language). These descriptions are used in automated overview pages (as e.g. in /mmbase/validation). If you define a data type for inclusion in mmbase.jar or in some component, it is advised to at least provide a description in English.

There can be one or no 'class' element then, which defines the actual implementation of the org.mmbase.datatypes.DataType interface which backs the data type. The class can often be inherited from the base data type. If not, the class must at least extend from the class from the base data type.

With every data type a default value is associated. E.g.

<datatype id="integer">
    <specialization id="colors">
      <description xml:lang="en">
        colors as can be used in css
      </description>
      <default value="white" />
      <enumeration>
        <entry basename="org.mmbase.datatypes.resources.string.colors" />
      </enumeration>
    </specialization>

The value is implicitly cast to the correct type. In this case 'white' can be converted to an integer, because there is a mapping present in the defined enumeration.

If the class of the data type supports properties which are not recognized by the data type XSD (as described in this document) then arbitrary properties can be set using 'property' sub elements.

<specialization id="confirmpassword">
  <description xml:lang="en">
    This is a field datatype that of which the value is only valid, if it is equal to the value
    of another field in this node. The other field's name is default 'password' but it can be
    changed by overriding the property 'field'.
  </description>
  <class name="org.mmbase.datatypes.ConfirmPasswordDataType" />
  <property name="field" value="password" />
</specialization>

In this case, the 'ConfirmPasswordDataType' class has a 'setField' method which is called when this data type is instantiated.

<specialization id="emailaddress">
  <description xml:lang="en">
    The valid values of such a field must be a valid email address. The current implemenation is by a rather simple regular expression.
   </description>
   <pattern value="(?i)\A[A-Z0-9_\-\+\&amp;\*\#]+([\.-]?[A-Z0-9_\-\+\&amp;\*\#])*@([A-Z0-9_-]{2,}[\.]{1})+([A-Z]{2,})\z">
     <description xml:lang="nl">'${VALUE}' is geen geldig email adres</description>
     <description xml:lang="en">'${VALUE}' is not a valid email-address</description>
     <description xml:lang="fr">'${VALUE}' n'est pas un correct address courriel</description>
     <description xml:lang="eo">'${VALUE}' ne estas korektforma retadreso</description>
   </pattern>
</specialization>

As you see, token replacement is done. Currently, NAME, VALUE, CONSTRAINT and CONSTRAINTVALUE are recognized. No java.text.MessageFormat features are yet supported.

2.2.3. Unique

A value can be marked to be 'unique'. This can only be applied if the data type is associated with a field (See also Section 3, “How to associate a data type with a field”) Of course this means that that values which are already present, are not valid any more for new objects.

2.2.4. Required

If a data type is 'required' then that means that the values may not be null.

2.2.5. Minimal and maximal values

For datatypes which describe 'comparable' values (and so the class implements org.mmbase.datatypes.ComparableDataType), a minimal and maximal value can be assigned.

<datatype base="datetime" xmlns="http://www.mmbase.org/xmlns/datatypes">
   <default value="now" />
   <minExclusive value="today - 3 day"    enforce="oncreate" />
   <maxInclusive value="today + 100 year" enforce="onchange" />
</datatype>

As you can see, these minimal and maximal values can be set -Exclusive or -Inclusive, with the obvious semantics.

Note

Most datatypes are comparable. But e.g. not binaries or xml's. It is most logical for dates, and numerics. For strings it indicates alphabetic ordering (I have yet to encounter the first serious use of a restriction on minimal or maximal value of a string field)

2.2.6. Minimal, maximal and fixed lengths

The values of some data types have a length (implementations of org.mmbase.datatypes.LengthDataType). This length can also be restricted. Noticeable 'length data types' are strings and binaries. The 'maximal length' can also be used as an indication for the length of the field in the database (if the database is associated with a field).

<datatype base="binary" xmlns="http://www.mmbase.org/xmlns/datatypes">
  <minLength value="3" />
  <maxLength value="10000" />
</datatype>

If minLength and maxLength are equal, you can simply use 'length'.

Note

A minimal length of '1' is often a bit confusing, because that means approximately the same as 'required'. Especially when using taglib generated input-fields. Taglib interprets the empty string as 'null' if the value is not required. So, it is adviced to use the 'required' restriction if you find that content for this data type is essential.

2.2.7. Patterns

For strings, the pattern is interpreted as a regular expression, which must match. For date times, the pattern is interpreted as SimpleDataFormat pattern (but see also org.mmbase.datatypes.DateTimePattern).

2.3. Other properties & enumerations

2.3.1. Password

For strings you can set the password to 'true', which is only an indication to user interfaces. So this is not a restriction, it is more like a property (and actually, you can achieve the same effect with a property element).

<specialization id="password">
  <pattern enforce="never" value="\A[^\n\r]*\z" /> <!-- cannot contain newlines -->
  <password value="true" />

  ..

  this would work too:
<specialization id="password">
   <property name="password" value="true" ?>
   <pattern enforce="never" value="\A[^\n\r]*\z" />
   ..

2.3.2. Enumerations

It is also possible to restrict the values to the entries of a certain set. There are several ways to provided those certain values. The most basic one, is simply stating every value:

<specialization id="spells">
<!-- example to test -->
  <enumeration>
    <entry value="1" xml:lang="nl" display="hokus pokus" />
    <entry value="2" xml:lang="nl" display="simsalabim" />
    <entry value="3" xml:lang="nl" display="pilatus pas" />
    <entry value="4" xml:lang="nl" display="abcracadabra" />
    <entry value="2" xml:lang="en" display="simsalabim" />
    <entry value="3" xml:lang="en" display="pilatus pas" />
    <entry value="1" xml:lang="en" display="hocus pocus" />
  </enumeration>
</specialization>

You can see that you can enter the same value more than once, for different languages. The default value to display is the value itself, so it is not needed to specify localized display values.

Enumerations are most often used on data types with 'base="integer"', but can also work on string data types, and node data type (using 'query').

An enumeration is a restriction, but or course it is mainly a powerfull hint to produce a user interface using e.g. a dropdown list or so.

Enumeration from a Resource bundle

It is also possible to specify the possible value using a ResourceBundle.

<enumeration >
   <entry basename="org.mmbase.datatypes.resources.weekdays" />
</enumeration>

where the weekdays resource bundle could look something like this:

0=sunday
1=monday
2=tuesday
3=wednesday
4=thursday
5=friday
6=saturday

If you look in the datatypes.xml in mmbase.jar you will find that this 'weekdays' enumeration actually defined like this

<enumeration>
   <entry basename="org.mmbase.datatypes.resources.weekdays"
          javaconstants="java.util.Calendar"
/>
</enumeration>

This indicates that the keys of the resource bundle are resolved to the actual values using constants in the given java class. This results in that the resource bundle can look like this:

SUNDAY=sunday
MONDAY=monday
TUESDAY=tuesday
WEDNESDAY=wednesday
THURSDAY=thursday
FRIDAY=friday
SATURDAY=saturday

Which looks of course a lot nicer.

sorterclass attribute

To specify an alternative presentation order, you can specify a 'Comparator' or 'Comparable' class name with the 'sorterclass' attribute. This will be used to order the entries, using the keys. If it is a comparator, then it will need the empty constructor, if it is a comparable, then it will be used to 'wrap' the keys, so a constructor must exist, which can accept the keys as an argument.

<datatype base="long" xmlns="http://www.mmbase.org/xmlns/datatypes">
  <enumeration>
    <entry
       sorterclass="org.mmbase.util.ReverseComparator"
       basename="org.mmbase.notifications.resources.offset" />
  </enumeration>
      </datatype>
TUESDAY=tuesday
WEDNESDAY=wednesday
THURSDAY=thursday
FRIDAY=friday
SATURDAY=saturday

A special, and relatively often, value for the value of 'sorterclass' is org.mmbase.util.SortedBundle$ValueWrapper. Using this, the entries will be orderd by (localized) value. This will often be used for enumeration which have no intrincic order. Drop downs will then be presented in alphabetical order. For example, This gives all languages:

<enumeration>
   <entry sorterclass="org.mmbase.util.SortedBundle$ValueWrapper"
          basename="org.mmbase.datatypes.resources.iso639" />
</enumeration>

Enumerations based on MMBase queries

TODO: query sub element.

2.4. Processors

The data type also provides hooks to plug in functionality on several events like 'getting a value' 'setting a value' and 'committing the object which contains this data type'. This is at the moment only sensible for fields. MMBase provides several general purpose processors, which you can also use in your own datatypes. But of course you can also implement new ones (if you want to do that, we refer to the javadoc of the org.mmbase.datatype.processors package)

2.4.1. Set-processors and Get-processors

Lets take an example:

<specialization id="created">
  <description xml:lang="en">
    Read-only date-time field that contains the time of creation of the node.
  </description>
  <default value="now" />
  <setprocessor>
     <class name="org.mmbase.datatypes.processors.Readonly" />
  </setprocessor>
  <getprocessor type="*">
    <class name="org.mmbase.datatypes.processors.CreationTimeGuesser" />
  </getprocessor>
  <getprocessor type="string">
    <class name="org.mmbase.datatypes.processors.CreationTimeGuesser" />
    <class name="org.mmbase.datatypes.processors.FormatDateTime" />
  </getprocessor>
</specialization>

Using the 'type' attribute one of the MMBase database type can be associated with get- and setprocessors. Those database types also correspond to methods in the MMBase bridge 'Node' interface. In this case we see that when getting a value as a string ('getStringValue'), something else happens then when it is gotten otherwise ('getDateValue'). In this case the creation time is nicely formatted when the value is requested as a string.

The CreationTimeGuesser processor arranges that if the value is empty, it guesses an actual value using the MMBase 'day markers'.

In this example, there is also hooked in a class on the set-processor. Namely the processor which simply refuses the action, effectively leading to a read-only field. (The CreationTimeGuesser itself can use 'setObjectValue' to bypass all set-processors, if it wants to store its guessed value for efficiency).

Note

You can specify more than one class for a certain processor. They will be 'chained' in such a case.

Note

The class may also implemented org.mmbase.util.transformers.CharTransformer (or factory), in which case an instance of that class is simply wrapped into a Processor, which only transforms the value.

Note

The instances of the processors can be configured with 'param' subtags.

2.4.2. Commit-processors

Commit-processors are effectively called on commit of the node which contains the field with this data type (for other datatypes the behavior is as yet undefined). For example

<specialization id="lastmodified">
  <description xml:lang="en">
    Read-only date-time field that contains the most recent time at which this node was changed .
  </description>
  <setprocessor>
    <class name="org.mmbase.datatypes.processors.Readonly" />
  </setprocessor>
  <commitprocessor>
    <class name="org.mmbase.datatypes.processors.LastModified" />
  </commitprocessor>
  ..

So, on commit, the 'LastModified' processor is called, which simply fills the current time into the value of the field with this data type.

3. How to associate a data type with a field

A data type specialization needs to be added to a field in a builder configuration file just before the 'db' section. By the way, since the datatypes can provide the necessary information, the db section can be omitted all together. The following example creates a lastmodified field for a node that is automatically adjusted every time a node is saved.

<field>
  <datatype xmlns="http://www.mmbase.org/xmlns/datatypes" base="lastmodified" />
  <db>
    <name>updated</name>
    <type key="false" notnull="false" state="system">DATETIME</type>
  </db>
</field>

which can also be noted like this.

<field name="updated" state="system">
  <datatype xmlns="http://www.mmbase.org/xmlns/datatypes" base="lastmodified">
    <required value="false" /> <!-- this is btw also the default -->
  </datatype>
</field>

Note

Using 'db' subtag is actually a MMBase pre-1.8 remnant. Since MMBase 1.8 the db type of a field can be completely implicit by the data type, and the remaining properties of the db-section were not really database related. E.g. the name of a field is much more generic, and is used not only in the database layer. The database layer may even use something else. The 'state' of a field is only partially database related. So 'name' and 'state' are now also attributes of the field tag itself.

Keys can be indicated in a seperate 'indexlist' section of the builder xml.

The database property 'notnull' can correspond to the 'required' restriction of the data type.

As you can see, this is simply a part of the same XML as also used in datatypes.xml's. So, you can also 'anonymously' define or refine a data type just for one field.

4. Datatypes and the MMBase bridge

When you use the bridge, you can get the data type associated with a certain field use the 'getDataType' method on 'Field'. Then you can use the interfaces of 'org.mmbase.datatypes' to explore the possible values.

5. Datatypes and the MMBase Taglib

If the mm:fieldinfo tag is used to create form-entries, then the data type of the field is explored to do that. If it e.g. detects that new-lines are permissible, then a textarea is created.

5.1. Forms & validation

TODO: mm:input, mm:form, mm:valid, mm:datatype

6. Datatypes and JavaScript

Since MMBase 1.8.5, the meta information provided by MMBase Data Types can also be accessed using JavaScript. For that you include the javascript file /mmbase/validation/validation.js.jsp, plus some other utilities (see /mmbase/valiation/test.jspx), and instatiate an "MMBaseValidator".

<html  xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <title>Test</title>
    <jsp:directive.include file="/mmbase/validation/javascript.jspxf" />
  </head>
  <body>
    <script type="text/javascript">new MMBaseValidator(window);</script>
    ... the rest of your page..

This javascript library searches (on load) all form elements with the class "mm_validate". 'onkeyup' events are added to all those elements.

The data type is determined by inspecting the css class names. If one starts with "mm_dt_" then the remaining is interpreted as the MMBase Data Type id. There can also be two class names, one starting with mm_f_ and mm_nm_ which can together identify the datatype of a certain field of a nodemanager.

The mm:fieldinfo tag of the MMBase Taglib will set up the correct CSS class for form elements that it creates.

As soon a data type meta information is needed, then this is requested (using 'ajax') with a call to /mmbase/validation/datatype.jspx. The resulting XML's are cached in MMBaseValidator.

7. Summary of default datatypes

This is a summary of all MMBase datatypes and their specializations found in the 'datatypes.xml' in mmbase.jar and as such are per default available. The basic data types (the 'database types') are xml, string, datetime, long, boolean, integer, long, float, double, binary and node, below their respective titles you will find their specializations.