Pentaho Analysis Services: Developer's guide

Developer's Guide

There are several ways to get Mondrian running. The easiest is to download a binary release, as described in the installation guide. But you can also build Mondrian from its source code. This document describes how to do that, how to learn about Mondrian's inner workings, and the guidelines you'll need to follow if you want to contribute to the Mondrian project.

Getting the source code

First, you need to get a copy of the source code. You can get the source code from SourceForge or from the project's Perforce source server.

Download the latest release

Download the latest mondrian-version.zip from SourceForge, and unzip. Now find the mondrian-version-src.zip inside this distribution, and unzip it. The directory you unzip this source code to — typically something like C:\open\mondrian or /usr/local/mondrian-x.y.z — will be denoted ${project.location} later in this document.

Connect to the Perforce source-code server

If you are a mondrian developer, and need to access the latest source code and check in changes, you should connect to the Perforce source-code server. If you are not a developer, but are interested in getting the latest code, you can connect as the 'guest' user.

Download the perforce client from http://www.perforce.com/perforce/loadprog.html
If you have Windows, start the perforce UI (P4Win). (If you are not running windows, you will have to use the command-line interface to do the following; 'p4 help' should get you started.)
Choose Settings > Switch Port Client User...; the following dialog appears.
Set Server to perforce.eigenbase.org; Port to 1666; set User to your SourceForge username (usually guest, unless you are a commiter to the project); set Client to your username plus the name of your machine (for example, guest.jhyde.stilton).
Choose ClientSpec > New... and create a client with the same name. Set its root to something like 'C:\' or 'D:\work', and view specification of
```
//open/mondrian/... //<<clientname>>/mondrian/...
```
Sync to head revision.

Checking in code

If you are a regular contributor to the Mondrian project, we will give you privileges to commit to the source code server. As a commiter, you will be able to add, edit, and delete files in the source system, and commit changelists. Usually we ask you to prove your worth with a few tasks before welcoming you to the team; contact Julian Hyde for more information on how to join the team.

When you check in:

Write a unit test for your change. (Or unit tests: the number of unit tests you write should be appropriate for the scale of your change.) The test should fail before you make the change, and succeed after it. We recommend a test-driven development process, where you write the test before you change the code. Unit tests are particularly important if you are fixing a bug.
Run the regression suite, and make sure all tests pass. We don't expect you to run the suite in all configurations (DBMSes, operating systems, JDK versions, and parameter settings) but if your change affects something configuration-specific, be smart and test more than one configuration. For example, SQL generation is DBMS-specific, and file handling is OS-specific, but MDX function implementations are typically generic.
Make sure your change adheres to the coding guidelines.
If the change affects mondrian's user-visible behavior, update the documentation. For example, if you make a schema change, update schema.html.
If your change affects a public API, discuss the change with mondrian developers first. We want to preserve backwards compatibility if possible, or at least clearly document the change in the release notes.
Check in your changes in a single perforce changelist. The changelist must have a helpful description, prefixed by 'MONDRIAN: ' to distinguish it from changes to other projects on the same server. If your changes fixes a bug, reference the bug number in your changelist description, and reference the changelist number and expected release number in the bug comments when closing the bug.

Building the code

Next, install a build environment. Install the JDK, Ant, Tomcat, Xalan, and JUnit, and set JAVA_HOME, ANT_HOME, TOMCAT_HOME, XALAN_HOME, JUNIT_HOME in your environment.

Download the latest jpivot-version.war from the JPivot project at SourceForge and save it as ${project.location}/lib/jpivot.war.

Now build the code, as follows:

C:\mondrian> ant Buildfile: build.xml sniff: prepare: parser: [javacup] Opening files... [javacup] Parsing specification from C:\mondrian\src\main\mondrian\olap\Parser.cup... [javacup] Checking specification... [javacup] Warning: Terminal "UNKNOWN" was declared but never used [javacup] Warning: Non terminal "unsigned_integer" was declared but never used [javacup] Building parse tables... [javacup] Computing non-terminal nullability... [javacup] Computing first sets... [javacup] Building state machine... [javacup] Filling in tables... [javacup] Checking for non-reduced productions... [javacup] Writing parser... [javacup] Closing files... [javacup] ------- CUP v0.10k Parser Generation Summary ------- [javacup] 0 errors and 2 warnings [javacup] 47 terminals, 49 non-terminals, and 100 productions declared, [javacup] producing 153 unique parse states. [javacup] 2 terminals declared but not used. [javacup] 0 non-terminals declared but not used. [javacup] 0 productions never reduced. [javacup] 0 conflicts detected (0 expected). [javacup] Code written to "Parser.java", and "ParserSym.java". [javacup] ---------------------------------------------------- (v0.10k) generate.resources: [javac] Compiling 2 source files to D:\open\mondrian\classes [resgen] Generating D:\open\mondrian\src\main\mondrian\olap\MondrianResource.java [resgen] Generating D:\open\mondrian\src\main\mondrian\olap\MondrianResource.properties [resgen] Generating D:\open\mondrian\src\main\mondrian\olap\MondrianResource_en_US.java [resgen] Generating D:\open\mondrian\src\main\mondrian\olap\MondrianResource_en_US.properties [resgen] Generating D:\open\mondrian\src\main\mondrian\olap\MondrianResource_de_DE.java [resgen] Generating D:\open\mondrian\src\main\mondrian\olap\MondrianResource_de_DE.properties def: [xomgen] Writing src\main\mondrian\olap\mondrian.dtd [xomgen] Writing src\main\mondrian\olap\MondrianDef.java [xomgen] Done [copy] Copying 1 file to D:\open\mondrian\lib compile.java: [javac] Compiling 791 source files to D:\open\mondrian\classes [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -deprecation for details. compile.jsp.maybe: copy.properties: [copy] Copying 4 files to D:\open\mondrian\classes compile: BUILD SUCCESSFUL Total time: 46 seconds

Installing the database

Before you run the regression test suite or the web application, you must install the standard FoodMart dataset. This is described in the installation guide.

If you got your files from the Perforce server, you can skip the step where you download the data sets: you should already have the files demo/access/MondrianFoodMart.mdb and demo/FoodMartCreateData.zip.

Running the test suite

At the command line:

cd ${project.location} ant test

Running the test via the Mondrian Ant build in Eclipse works, too.

Example output:

Buildfile: build.xml Overriding previous definition of reference to jdk prepare: [mkdir] Created dir: C:\open\mondrian\build parser: [javacup] Opening files... [javacup] Parsing specification from C:\open\mondrian\src\main\mondrian\olap\Parser.cup... [javacup] Checking specification... [javacup] Warning: Terminal "UNKNOWN" was declared but never used [javacup] Warning: Non terminal "unsigned_integer" was declared but never used [javacup] Building parse tables... [javacup] Computing non-terminal nullability... [javacup] Computing first sets... [javacup] Building state machine... [javacup] Filling in tables... [javacup] *** Reduce/Reduce conflict found in state #99 [javacup] between value_expression_primary ::= NULL (*) [javacup] and term3 ::= term3 IS NULL (*) [javacup] under symbols: {EOF, AND, AS, CELL, DIMENSION, ELSE, END, IN, IS, MATCHES, MEMBER, NOT, ON, OR, PROPERTIES, SELECT, SE T, THEN, WHEN, XOR, COLON, COMMA, EQ, GE, GT, LE, LT, NE, RBRACE, RPAREN} [javacup] Resolved in favor of the second production. [javacup] *** Shift/Reduce conflict found in state #99 [javacup] between value_expression_primary ::= NULL (*) [javacup] under symbol EOF [javacup] Resolved in favor of shifting. ... [javacup] *** Shift/Reduce conflict found in state #99 [javacup] between term3 ::= term3 IS NULL (*) [javacup] under symbol RPAREN [javacup] Resolved in favor of shifting. [javacup] Checking for non-reduced productions... [javacup] Writing parser... [javacup] Closing files... [javacup] ------- CUP v0.10k Parser Generation Summary ------- [javacup] 0 errors and 63 warnings [javacup] 56 terminals, 60 non-terminals, and 125 productions declared, [javacup] producing 194 unique parse states. [javacup] 2 terminals declared but not used. [javacup] 0 non-terminals declared but not used. [javacup] 0 productions never reduced. [javacup] 61 conflicts detected (61 expected). [javacup] Code written to "Parser.java", and "ParserSym.java". [javacup] ---------------------------------------------------- (v0.10k) generate.resources: [javac] Compiling 3 source files to C:\open\mondrian\classes [resgen] Generating C:\open\mondrian\src\main\mondrian\resource\MondrianResource.java [resgen] Generating C:\open\mondrian\classes\mondrian\resource\MondrianResource.properties [resgen] Generating C:\open\mondrian\src\main\mondrian\resource\MondrianResource_en_US.java [resgen] Generating C:\open\mondrian\classes\mondrian\resource\MondrianResource_en_US.properties [resgen] Generating C:\open\mondrian\src\main\mondrian\resource\MondrianResource_de_DE.java [resgen] Generating C:\open\mondrian\classes\mondrian\resource\MondrianResource_de_DE.properties [resgen] Generating C:\open\mondrian\src\main\mondrian\resource\MondrianResource_de.java [resgen] Generating C:\open\mondrian\classes\mondrian\resource\MondrianResource_de.properties [resgen] Generating C:\open\mondrian\src\main\mondrian\resource\MondrianResource_es_ES.java [resgen] Generating C:\open\mondrian\classes\mondrian\resource\MondrianResource_es_ES.properties def: [xomgen] Writing src\main\mondrian\olap\mondrian.dtd [xomgen] Writing src\main\mondrian\olap\MondrianDef.java [xomgen] Done [copy] Copying 1 file to C:\open\mondrian\lib [copy] Copying 1 file to C:\open\mondrian\lib [xomgen] Writing src\main\mondrian\xmla\datasourcesconfig.dtd [xomgen] Writing src\main\mondrian\xmla\DataSourcesConfig.java [xomgen] Done [copy] Copying 1 file to C:\open\mondrian\lib compile.java: [javac] Compiling 987 source files to C:\open\mondrian\classes [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. check-FoodMartCreateData-uptodate: unzip-FoodMartCreateData: [unzip] Expanding: C:\open\mondrian\demo\FoodMartCreateData.zip into C:\open\mondrian\demo check-FoodMartAccessDB-uptodate: unzip-FoodMartAccessDB: [unzip] Expanding: C:\open\mondrian\demo\access\MondrianFoodMart-Access.zip into C:\open\mondrian\demo\access check-FoodMartDerbyDB-uptodate: unzip-FoodMartDerbyDB: [unzip] Expanding: C:\open\mondrian\demo\derby\derby-foodmart.zip into C:\open\mondrian\demo\derby compile: compile.tests: [javac] Compiling 69 source files to C:\open\mondrian\testclasses [javac] Note: C:\open\mondrian\testsrc\main\mondrian\test\ParameterTest.java uses or overrides a deprecated API. [javac] Note: Recompile with -deprecation for details. test-dbms: [echo] Connecting to jdbc:postgresql://localhost/FM3 [java] Mondrian: properties loaded from 'file:/C:/open/mondrian/mondrian.properties' [java] Mondrian: properties loaded from 'file:/C:/open/mondrian/build.properties' [java] Mondrian: loaded 4 system properties [java] testName: null [java] testClass: null [java] All 1 thread(s) started. [java] Mondrian: JDBC driver org.postgresql.Driver loaded successfully [java] Mondrian: JDBC driver sun.jdbc.odbc.JdbcOdbcDriver loaded successfully [java] Mondrian: JDBC driver com.mysql.jdbc.Driver loaded successfully [java] Mondrian: JDBC driver oracle.jdbc.OracleDriver loaded successfully [java] [0] ........................................ [java] [40] ........................................ [java] [80] ........................................ [java] [120] ........................................ [java] [160] ........................................ [java] [200] ........................................ [java] [240] ........................................ [java] [280] ........................................ [java] [320] ........................................ [java] [360] ........................................ [java] [400] ........................................ [java] [440] ........................................ [java] [480] ........................................ [java] [520] ........................................ [java] [560] ........................................ [java] [600] .................. [java] OK (618 tests) [java] Time: 711.63 [java] Normal termination. BUILD SUCCESSFUL Total time: 12 minutes 13 seconds

Create, deploy and start the web application

At the command prompt, type

ant war

This will create lib/mondrian.war. Copy mondrian.war to the TOMCAT_HOME/webapps directory.

Now, start Tomcat and hit http://localhost:8080/mondrian.

Coding guidelines

If you are contributing code, please follow the same guidelines used for the rest of the code. ("When in Rome, do as the Romans do.")

Code content:

Declare variables as near to their first use as possible.
Don't initialize variables with 'dummy' values just to shut up the compiler.
One declaration per line is recommended.
Only one top-level class should be defined per java file.

Documentation and comments:

Source files must contain copyright and license notices.
Classes and public methods must have javadoc.
Write Javadoc comments on methods in the present active ('Collects garbage.'), not the imperative ('Collect garbage.'), passive ('Garbage is collected.'), or future active ('Will collect garbage.').
When editing HTML documents, please don't use an editor which reformats the HTML source (such as Microsoft Word).

Spacing and indentation:

Use spaces, not tabs.
Indentation 4.
Open braces on the same line as the preceding 'if', 'else', 'while' statement, or method or 'class' declaration.
Use braces even for single-line blocks.
Try to keep lines shorter than 80 characters.

The following images show my code style settings in IntelliJ. If you use IntelliJ, plug in these settings; if not, they should give you an idea of the code formatting policy.

Code formatting: General

Code formatting: Alignment and Braces

Code formatting: Wrapping

Code formatting: Spaces

Learning more about Mondrian

How Mondrian generates SQL

If you're feeling mystified where the various SQL statements come from, here's a good way to learn more. Give it a try, and if you have more questions I'll be glad to answer them.

In a debugger, put a break point in the RolapUtil.executeQuery() method, and run a simple query. The easiest way to run a query is to run a junit testcase such as BasicQueryTest.testSample0(). The debugger will stop every time a SQL statement is executed, and you should be able to loop up the call stack to which component is executing the query.

I expect that you will see the following phases in the execution:

One or two SQL queries will be executed as the schema.xml file is read (validating calculated members and named sets, resolving default members of hierarchies, and such)
A few SQL queries will be executed to resolve members as the query is parsed. (For example, if a query uses [Store].[USA].[CA], it will look all members of the [Store Nation] level, then look up all children of the [USA] member.)
When the query is executed, the axes (slicer, columns, rows) are executed first. Expect to see more queries on dimension tables when expressions like [Product].children are evaluated.
Once the axes are populated, the cells are evaluated. Rather than executing a SQL query per cell, Mondrian makes a pass over all cells building a list of cells which are not in the cache. Then it builds and executes a SQL query to fetch all of those cells. If it didn't manage to fetch all cell values, it will repeat this step until it does.

Remember that the purpose of these queries is to populate cache. There are two caches. The dimension cache which maps a member to its children, e.g.

[Store].[All Stores] → { [Store].[USA], [Store].[Canada], [Store].[Mexico]}

The aggregation cache maps a tuple a measure value, e.g.

([Store].[USA], [Gender].[F], [Measures].[Unit Sales]) → 123,456

Once the cache has been populated, the query won't be executed again. That's why I recommend that you restart the process each time you run this in the debugger.

Appendix A. Product installation instructions

These are the products I used to build mondrian. Install all of the products marked 'required'.

In the following, the symbol ${project.location} means the root of your source tree.

Product	Required?	Version	Comment
JDK	Yes	1.5 or later (1.4 support available via retroweaver)	Available from http://developer.java.sun.com/. I downloaded `jdk-1_5_0_08-windows-i586-p.exe`, and installed to `C:/jdk1.5.0_08`, and set `JAVA_HOME` to the same.
Ant	Yes	1.6 or later	Available from http://ant.apache.org/bindownload.cgi. I downloaded `apache-ant-1.6.2-bin.zip`, extracted to `C:/jakarta-ant-1.6.2`, and set `ANT_HOME` to the same.
Tomcat	Yes	5.0.25 or later	Available from http://jakarta.apache.org/tomcat. I downloaded `jakarta-tomcat-5.0.25.zip`, extracted to `C:/jakarta-tomcat-5.0.25`, and set `TOMCAT_HOME` to the same.
Xerces		Use the version included with Tomcat	Xerces is included with Tomcat. If you use a different version, compatibility issues may arise.
Xalan	Yes	2.6.0 or later	Available from http://xml.apache.org/xalan-j/. I downloaded `xalan-j_2_6_0-bin.zip`, extracted to `C:/xalan-j_2_6_0`, and set `XALAN_HOME` to the same. Important: copy `XALAN_HOME``/bin/xalan.jar` to `TOMCAT_HOME/common/endorsed/`.
JUnit	Yes	3.8.1 or later	Available from http://www.junit.org/. I downloaded `junit3.8.1.zip`, extracted to `C:/junit3.8.1`, and set `JUNIT_HOME` to the same.
JavaCUP (parser generator)	Included with source distribution, as `lib/javacup.jar`.	v0.10g (with modifications)	Available from http://www.cs.princeton.edu/~appel/modern/java/CUP/. I modified version v.0.10g to add an Ant task, and to output error messages in a format which Emacs can parse.

Author: Julian Hyde; last updated December, 2007.
Version: $Id: //open/mondrian-release/3.0/doc/developer.html#2 $ (log )
Copyright (C) 2005-2007 Julian Hyde