Data Loaders

    
  Prev  Next  Contents  Home
 

Page contents

Alphabetical index of keys:

Introduction

Data loaders are Java objects that load data from certain type of data source, such as a CSV file or MySQL data-base, and expose the data as variables for the templates. For example, a data loader was invoked in the Quick Tour when you have written csv(data/birds.csv).

Data loaders are typically invoked:

Predefined data loaders

Data loaders that load data directly from files will want you to give the path of the file as parameter. There you have to give real path. If you give a relative path, then it will be interpreted relatively to the dataRoot, which defaults to the sourceRoot if you didn't specified it. The data files can be outside the dataRoot directory; it is used only as a base directory.

Note: I didn't written database/SQL/JDBC data loader yet... Of coruse, you can write a such data loader yourself. Contributions are welcome!

csv

Parameters:
  1. path: string. The path of the CSV file.
  2. options: hash, optional. The list of valid options:

Examples (with TDD syntax):

The csv data loader parses CSV (Column Separated Values) file, or other file of similar formats (as tab divided text, comma separated values), and returns it as a sequence. The sequence is the list of the table rows, and each row is a hash, where you can access the cells with the column name. The column names (headers) are stored in the first row of the CSV file (unless you use the headers option), so this row is not part of the result sequence.

For example, if this is the CSV file:

name;color;price
rose;red;10
rose;yellow;15
tulip;white;6

and you load it into the variable flowers, then you can print it as:

<table border=1>
<#list flowers as flower>
  <tr><td>${flower.name}<td>${flower.color}<td>${flower.price}
</#list>
</table>

and the output will be:

<table border=1>
  <tr><td>rose<td>red<td>10
  <tr><td>rose<td>yellow<td>15
  <tr><td>tulip<td>white<td>6
</table>

The rows are not only hashes, but also sequences, so instead of the column name you can use the 0 based column index. Thus the above template could be written like this as well, just it is less readable:

<table border=1>
<#list flowers as flower>
  <tr><td>${flower[0]}<td>${flower[1]}<td>${flower[2]}
</#list>
</table>

and actually then it would be simpler to write:

<table border=1>
<#list flowers as flower>
  <tr><#list flower as cell><td>${cell}</#list>
</#list>
</table>

The values (cells) in the CSV file will be always exposed as string variables, unless you specify a different type in the header cells directly. To specify the type, type a colon after the column name, and a type identifier. The type identifier can be: n (or number) or b (or boolean) or d (or date) or t (or time) or dt (or dateTime) or s (or string). For example, if you want to expose the price column as number, and not as string (so you can do arithmetic with it, or use the number formatting facilities), then the CSV file would be:

name;color;price:n
rose;red;10
rose;yellow;15
tulip;white;6

Numerical values must use dot (.) as decimal separator, and no grouping, unless you change these with the decimalSeparator and/or groupingSeparator options.

Boolean values must be one of true, yes, y, 1, false, no, n, 0, or the words defined be the altTrue and altFalse options. Upper- and lower-case letters are not distinguished.

Date, time and date-time values use common SQL format with optional time zone, as "2003-06-25", "22:30:08", "2003-06-25 10:30:08 PM", "2003-06-25 10:30:08 PM GMT+02:00". But if you use option dateFormat, timeFormat, or datetimeFormat, then that format has to be used for dates, times, and date-times respectively. If the time zone is not given in a value, the value of the timeZone setting is used.

The variable returned by the csv data loader is not only a sequence, but also a hash at the same time, that contains one key: headers. This is a sequence that stores the column names.

<#list flowers.headers as h>
- ${h}
</#list>

will print:

- name
- color
- price

Note that only the name of the column is returned, not the type identifier stuff (as :n).

text

Parameters:
  1. path: string. The path of the text file.
  2. charset: string, optional. The charset used for the text file. Defaults to the sourceEncoding setting.

Examples (with TDD syntax):

This data loader loads a plain text file, and returns that as a string.

slicedText

Parameters:
  1. path: string. The path of the text file.
  2. options: hash, optional. The list of valid options:

Examples (with TDD syntax):

This data loader loads a text file and slices it to items (strings) by cutting out all occurences of the separator string specified with the separator option. The result is a sequence of strings.

For example, if this is the data/stuff.txt file:

This is the first item.

Still the first item...

--8<--

The items are separated with this:
--8<--
just don't forget to surround it with
empty lines.

--8<--

This is the last item.

and it's loaded in the configuration file like: 

data: {
    stuff: slicedText(data/stuff.txt, {separator:"\n\n--8<--\n\n"})
}

then the output of this template file

<#escape x as x?html>

<#list stuff as i>
<pre>
${i}
</pre>
</#list>

</#escape>

will be this:

<pre>
This is the first item.

Still the first item...
</pre>
<pre>
The items are separated with this:
--8&lt;--
just don't forget to surround it with
empty lines.
</pre>
<pre>
This is the last item.
</pre>

Note the double \n-s in the separator that causes the data loader to treat "–8<--" as separator only if it is surrounded with empty lines.

tdd

Parameters:
  1. path: string. The path of the TDD file.
  2. charset: string, optional. The charset used for the TDD file. Defaults to the sourceEncoding setting.

Examples (with TDD syntax):

This data loader parses a TDD file. The loaded TDD is interpreted in hash mode, and TDD functions will invoke data loaders, exactly like with the data setting.

See <FMPP>/docs/examples/tdd for a concrete example.

tddSequence

Parameters:
  1. path: string. The path of the TDD file.
  2. charset: string, optional. The charset used for the TDD file. Defaults to the sourceEncoding setting.

This is like the tdd dataloader, except that it interprets the file as a TDD sequence (TDD "sequence mode"), rather than as a TDD hash. So the result will be a sequence (a list), not a hash.

xml

Parameters:
  1. path: string. The path of the XML file.
  2. options: hash, optional. Options. See later...

Note: Sometimes XML files are rather source files (as the HTML-s and the JPG in the Quick Tour) than data files (as data/birds.tdd in the Quick Tour), just you need to invoke a template to render them to their final form (e.g. to HTML pages). If this is the case, you should use renderXml processing mode (see here...), not XML data loader. An example that uses this approach is <FMPP>/docs/examples/xml_rendering.

Loads an XML file. This uses the built-in XML wrapper of FreeMarker 2.3 and later; please read FreeMarker Manual/XML Processing Guide for more information about the usage of the returned variable.

To use this, JAXP 1.2+ must be available. For more info, read the chapter about installing FMPP.

Notes:

The example <FMPP>/docs/examples/xml_try is a good tool to understand this data loader and its options.

Options:

eval

Parameters:
  1. expression: string. Expression in BeansShell language.
  2. variables: hash, optional. Values that will be visible for the expression as variables.

Evaluates a BeanShell expression, or runs a more complex BeanShell script and uses its return-ed value.

You can use the predefined variable engine to access the current FMPP engine instance.

Examples (with TDD syntax):

htmlUtils

Parameters: none

The returned hash contains custom directives that help HTML template development.

Currently it contains only 1 directive: img. This is used as HTML img element, but the width and height attributes are calculated automatically (unless these attributes are specified in the template). Also, the processing will stop with error, if the image file pointed by the src attribute is missing.

Example: If you have loaded the hash into the html variable, then you can do this in a template:

<@html.img src="falcon.png" alt="Falcon" />

and the output will be something like this (depending on the concrete dimensions of the image):

<img src="falcon.png" alt="Falcon" width="80" height="120">

See also the example in <FMPP>/docs/examples/img_dims.

Directive img accepts any extra parameters (attributes). The extra parameters will be printed as the attributes of the HTML img element.

xhtmlUtils

Parameters: none

This is the same as htmlUtils, but it is for XHTML output.

now

Parameters:
  1. options: hash, optional. The supported options are:

Examples (with TDD syntax):

This data loader loads the current date and time from the system clock, and produces a string from it. Since FreeMarker has introduced native date/time value support, it is maybe better to use pp.sessionStart or pp.now instead of this data loader.

get

Parameters:
  1. varName: string. The name of the variable.
  2. subVarName: string, optional. The name of the sub-variable.
  3. subSubVarName: string, optional. ...etc. You can specify any number of parameters.

This data loader returns the value of a variable that was already put into the data hash. For example:

data: {
    a: 123
    b: get(a)
}

Here the value of b will be the same as the value of a, which is 123. You can retrieve the value of subvariables with additional parameters:

data: {
    a: {
        x: 123
        y: {
            q:234
        }
    }
    b: get(a, x)
    c: get(a, y, q)
}

Here b will be 123, c will be 234.

The get data loader was introduced so that data loaders can get the values loaded by other data loaders previously. For example:

data: {
    doc: xml(data/foo.xml)
    index: com.example.MyIndexBuilderDataLoader(get(doc))
}

The get data loader sees the variables that were already put into the hash in the same enclosing data/localData hash TDD expression where it is called, but it doesn't see values that are put into the data model elsewhere, as in other (inherited or inheriting) configuration files. However, if you use get in the localData setting, it will also see the session level data (see data).

antProperty

Parameters:
  1. properyName: string. The name of the Ant property.
  2. defaultValue: any, optional. The value returned if the Ant property does not exist.

This data loader returns the value of an Ant property. If no Ant property with the given name exists, it will return nothing (i.e. will not add new variable to the shared data model), or it will return the value of the second parameter if you use that.

The values of Ant properties are strings. But sometimes you want to see a property as numerical variable, or as boolean variable, etc. If the property name you give as data loader parameter ends with ?n, then the string will be converted to number variable, and the ?n itself will not count as the part of the actual property name. The complete list of postfixes:

To see a concrete example, look at <FMPP>/docs/examples/ant2.

This data loader will work only if you execute FMPP as Ant task.

antProperites

Parameters:
  1. properyName1: string, optional. The name of an Ant property to expose.
  2. properyName2: string, optional. The name of another Ant property to expose.
  3. properyNameN: string, optional. ...etc. You can specify any number of Ant parameters.

Returns the set of all Ant properties, or the set of selected ant properties, as a hash.

If you use this data loader without parameters, then it returns the hash of all Ant properties. To see a concrete example, look at <FMPP>/docs/examples/ant.

You can also give the name of properties to expose as parameters. For example if you want to put only the Ant properties foo and bar into the shared data model, then you write: antProperties(foo, bar). Parameters that refer to not existing Ant properties will be silently ignored. The same postfixes (as ?n) are supported as with antProperty.

To see a concrete example, look at <FMPP>/docs/examples/ant2.

This data loader will work only if you execute FMPP as Ant task.

antProject

Parameters: none

Returns the current Ant project object (org.apache.tools.ant.Project). See the example in <FMPP>/docs/examples/ant.

This data loader will work only if you execute FMPP as Ant task.

antTask

Parameters: none

Returns the current Ant task object (org.apache.tools.ant.Task). See the example in <FMPP>/docs/examples/ant.

This data loader will work only if you execute FMPP as Ant task.

Custom data loaders

If you want to write custom data loader, you have to write a Java class that implements the fmpp.tdd.DataLoader interface (see in the API documentation). Then, if the class is available in the class path, or if you drop its jar into the lib directory of the FMPP installation, you can call it similarly as a predefined data loader, just you use the full-qualified class name as the data loader name. For example:
com.example.fmpp.SQLDataLoader("SELECT * FROM products").

If you have written a data loader that can be useful for other FMPP users, I will be happy to make it available on the FMPP home page for download. So if you want to contribute with your data loader, drop me a mail ([email protected] (delete the "REMOVETHIS"!)) or write to the mailing list. Thank you!

Prev  Next  Contents  Home      Report bug   Generated on Dec 16, 2007 10:12 PM GMT
For FMPP version 0.9.13
SourceForge Logo  Powered by FreeMarker