|
|
Boost.Jam |
Boost.Jam (BJam) is a build tool based on FTJam, which in turn is based on Perforce Jam. It contains significant improvements made to facilitate its use in the Boost Build System, but should be backward compatible with Perforce Jam.
This is version 3.1.10 of BJam and is based on version 2.4 of Jam/MR:
/+\ +\ Copyright 1993-2002 Christopher Seiwald and Perforce Software, Inc. \+/ This is Release 2.4 of Jam/MR, a make-like program. License is hereby granted to use this software and distribute it freely, as long as this copyright notice is retained and modifications are clearly marked. ALL WARRANTIES ARE HEREBY DISCLAIMED.
Jam is a make(1) replacement that makes building simple things simple and building complicated things manageable.
Jam's language is expressive, making Jamfiles (c.f. Makefiles) compact. Here's a sample:
Main smail : main.c map.c resolve.c deliver.c
misc.c parser.y alias.c pw.c headers.c
scanner.l getpath.c str.c ;
This builds "smail" from a dozen source files. Jam handles header file dependencies automatically and on-the-fly.
Jam is very portable: it runs on UNIX, VMS, Mac, and NT. Most Jamfiles themselves are portable, like the sample above.
Jam is unintrusive: it is small, it has negligible CPU overhead, and it doesn't create any of its own funny files (c.f. Odin, nmake, SunOS make).
Jam can build large projects spread across many directories in one pass, without recursing, tracking the relationships among all files. Jam can do this with multiple, concurrent processes.
Jam isn't under the blinkin GNU copyright, so you can incorporate it into commercial products.
| Jam.html | Jam and language reference. |
| Porting | Notes on porting jam to wildcat platforms. |
Installing BJam after building it is simply a matter of copying the generated executables someplace in your PATH. For building the executables there are a set of build bootstrap scripts to accomodate particular environments. The scripts take one optional argument, the name of the toolset to build with. When the toolset is not given an attempt is made to detect an available toolset and use that. The build scripts accept these areguments:
<build script name> [toolset]
Running the scripts without arguments will give you the best chance of success. On Windows platforms from a command console do:
cd <jam source location> .\build.bat
On Unix type platforms do:
cd <jam source location> sh ./build.sh
For the Boost.Jam source included with the Boost distribution the <jam source location> is BOOST_ROOT/tools/build/jam_src.
If the scripts fail to detect an appropriate toolset to build with your particular toolset may not be auto-detectable. In that case, you can specify the toolset as the first argument, this assumes that the toolset is readily available in the PATH. NOTE: The toolset used to build Boost.Jam is independent of the toolsets used for Boost.Build. Only one version of Boost.Jam is needed to use Boost.Build. The supported toolsets, and wether they are auto-detected, are:
| Script | Platforms | Toolsets | Detection |
|---|---|---|---|
| build.bat | Windows NT, 2000, and XP | como, Comeau.Computing C/C++ | |
| borland, Borland C++Builder (BCC 5.5) | * Common install location:
"C:\Borland\BCC55" * BCC32.EXE in PATH |
||
| gcc, GNU GCC | |||
| gcc-nocygwin, GNU GCC | |||
| intel-win32, Intel C++ Compiler for Windows | * ICL.EXE in PATH | ||
| metrowerks, MetroWerks CodeWarrior C/C++ 7.x, 8.x | * CWFolder variable configured * MWCC.EXE in PATH |
||
| mingw, GNU GCC as the MinGW configuration | * Common install location: "C:\MinGW" | ||
| msvc, Microsoft Visual C++ 6.x | * VCVARS32.BAT already configured * Common install locations: "C:\Program Files\Microsoft Visual Studio", "C:\Program Files\Microsoft Visual C++" * CL.EXE in PATH |
||
| vc7, Microsoft Visual C++ 7.x | * VCVARS32.BAT or VSVARS32.BAT
already configured * Common install location: "C:\Program Files\Microsoft Visual Studio .NET" * Common install location: "C:\Program Files\Microsoft Visual Studio .NET 2003" |
||
| build.sh | Unix, Linux, Cygwin, etc. | acc, HP-UX aCC | * aCC in PATH * uname is "HP-UX" |
| como, Comeau.Computing C/C++ | * como in PATH | ||
| gcc, GNU GCC | * gcc in PATH | ||
| intel-linux, Intel C++ for Linux | * icc in PATH * Common install locations: "/opt/intel/compiler70", "/opt/intel/compiler60", "/opt/intel/compiler50" |
||
| kcc, Intel KAI C++ | * KCC in PATH | ||
| kylix, Borland C++Builder | * bc++ in PATH | ||
| mipspro, SGI MIPSpro C | |||
| sunpro, Sun Workshop 6 C++ | * Standard install location: "/opt/SUNWspro" | ||
| true64cxx, Compaq C++ Compiler for True64 UNIX | |||
| vacpp, IBM VisualAge C++ | * xlc in PATH | ||
| MacOS X |
darwin, Apple MacOS X GCC | * uname is "Darwin" |
The built executables are placed in a subdirectory specific to your platform. For example, in Linux running on an Intel x86 compatible chip, the executables are placed in: "bin.linuxx86". The bjam[.exe] executable can be used to invoke Boost.Build.
The build scripts support additional invocation arguments for use by developers of Boost.Jam. The extra arguments come after the toolset, and can take the form of "--option" or targets for the build.jam script:
<build script name> [toolset] [--option+ target*]
There is current only one available option, "--debug", which builds debugging versions of the executable. When built they are placed in their own directory "bin.<platform>.debug". To specify targets without options, one can suply a special ignore option "---".
Currently there are two targets supported: dist, and clean. Respectively they: generate packages (compressed archives) as appropriate for distribution in the platform, or remove all the built executables and objects.
A number of enhancements have been made to the core language of Classic Jam. These changes were aimed primarily at making it easier to manage the complexity of a large system such as Boost.Build.
Classic Jam had an odd behavior with respect to command-line variable ( -s...) and environment variable settings which made it impossible to define an arbitrary variable with spaces in the value. Boost Jam remedies that by treating all such settings as a single string if they are surrounded by double-quotes. Uses of this feature can look interesting, since shells require quotes to keep characters separated by whitespace from being treated as separate arguments:
jam -sMSVCNT="\"\"C:\Program Files\Microsoft Visual C++\VC98\"\"" ...
The outer quote is for the shell. The middle quote is for Jam, to tell it to take everything within those quotes literally, and the inner quotes are for the shell again when paths are passed as arguments to build actions. Under NT, it looks a lot more sane to use environment variables before invoking jam when you have to do this sort of quoting:
set MSVCNT=""C:\Program Files\Microsoft Visual C++\VC98\""
The Boost.Build v2 initialization behavior has been implemented. This
behavior only applies when the executable being invoked is called
"bjam" or, for backward-compatibility, when the
BOOST_ROOT variable is set.
As a backward-compatibility measure for older versions of Boost.Build, when theBOOST_ROOTvariable is set, we first search forboost-build.jamin$(BOOST_ROOT)/tools/buildand$(BOOST_BUILD_PATH). If found, it is loaded and initialization is complete.
boost-build rule adds its (optional) argument to the
front of BOOST_BUILD_PATH, and attempts to load
bootstrap.jam from those directories. If a relative path is
specified as an argument, it is treated as though it was relative to the
boost-build.jam file.Boost Jam allows you to call a rule whose name is held in a variable or computed as the result of an expression:
x = foo ;
rule foobar { ECHO foobar ; } # a trivial rule
$(x)bar ; # invokes foobar
Furthermore, if the first expression expands to more than one list item, everything after the first item becomes part of the first argument. This allows a crude form of argument binding:
# return the elements of sequence for which predicate returns non-nil
rule filter ( sequence * : predicate + )
{
local result ;
for local x in $(sequence)
{
if [ $(predicate) $(x) ] { result += $(x); }
}
return $(result);
}
# true iff x == y
rule equal ( x y )
{
if $(x) = $(y) { return true; }
}
# bind 3 to the first argument of equal
ECHO [ filter 1 2 3 4 5 4 3 : equal 3 ] ; # prints "3 3"
You can now describe the arguments accepted by a rule, and refer to them by name within the rule. For example, the following prints ``I'm sorry, Dave'' to the console:
rule report ( pronoun index ? : state : names + )
{
local he.suffix she.suffix it.suffix = s ;
local I.suffix = m ;
local they.suffix you.suffix = re ;
ECHO $(pronoun)'$($(pronoun).suffix) $(state), $(names[$(index)]) ;
}
report I 2 : sorry : Joe Dave Pete ;
Each name in a list of formal arguments (separated by ``:'' in the rule declaration) is bound to a single element of the corresponding actual argument unless followed by one of these modifiers:
| Symbol | Semantics of preceding symbol |
|---|---|
| ? | optional |
| * | Bind to zero or more unbound elements of the actual argument. When ``*'' appears where an argument name is expected, any number of additional arguments are accepted. This feature can be used to implement "varargs" rules. |
| + | Bind to one or more unbound elements of the actual argument. |
The actual and formal arguments are checked for inconsistencies, which cause Jam to exit with an error code:
### argument error # rule report ( pronoun index ? : state : names + ) # called with: ( I 2 foo : sorry : Joe Dave Pete ) # extra argument foo ### argument error # rule report ( pronoun index ? : state : names + ) # called with: ( I 2 : sorry ) # missing argument names
If you omit the list of formal arguments, all checking is bypassed as in ``classic'' Jam. Argument lists drastically improve the reliability and readability of your rules, however, and are strongly recommended for any new Jam code you write.
Boost Jam introduces support for modules, which provide some rudimentary namespace protection for rules and variables. A new keyword, ``module'' was also introduced. The features described in this section are primitives, meaning that they are meant to provide the operations needed to write Jam rules which provide a more elegant module interface.
module expression { ... }
Code within the { ... } executes within the module named by evaluating expression. Rule definitions can be found in the module's own namespace, and in the namespace of the global module as module-name.rule-name, so within a module, other rules in that module may always be invoked without qualification:
module my_module
{
rule salute ( x ) { ECHO $(x), world ; }
rule greet ( ) { salute hello ; }
greet ;
}
my_module.salute goodbye ;
When an invoked rule is not found in the current module's namespace, it is looked up in the namespace of the global module, so qualified calls work across modules:
module your_module
{
rule bedtime ( ) { my_module.salute goodnight ; }
}
Each module has its own set of dynamically nested variable scopes. When execution passes from module A to module B, all the variable bindings from A become unavailable, and are replaced by the bindings that belong to B. This applies equally to local and global variables:
module A
{
x = 1 ;
rule f ( )
{
local y = 999 ; # becomes visible again when B.f calls A.g
B.f ;
}
rule g ( )
{
ECHO $(y) ; # prints "999"
}
}
module B
{
y = 2 ;
rule f ( )
{
ECHO $(y) ; # always prints "2"
A.g ;
}
}
The only way to access another module's variables is by entering that module:
rule peek ( module-name ? : variables + )
{
module $(module-name)
{
return $($(>)) ;
}
}
Note that because existing variable bindings change whenever a new
module scope is entered, argument bindings become unavailable. That explains
the use of "$(>)" in the peek rule above.
local rule rulename...
The rule is declared locally to the current module. It is not entered in the global module with qualification, and its name will not appear in the result of:
[ RULENAMES module-name ]
rule RULENAMES ( module ? )
Returns a list of the names of all non-local rules in the given module. If module is omitted, the names of all non-local rules in the global module are returned.
rule VARNAMES ( module ? )
Returns a list of the names of all variable bindings in the given
module. If module is omitted, the names of all variable bindings
in the global module are returned. Note:this includes any local
variables in rules from the call stack which have not returned at the time
of the VARNAMES invocation.
IMPORT allows rule name aliasing across modules:
rule IMPORT ( source_module ? : source_rules *
: target_module ? : target_rules * )
The IMPORT rule copies rules from the source_module into the target_module as local rules. If either source_module or target_module is not supplied, it refers to the global module. source_rules specifies which rules from the source_module to import; TARGET_RULES specifies the names to give those rules in target_module. If source_rules contains a name which doesn't correspond to a rule in source_module, or if it contains a different number of items than target_rules, an error is issued. For example,
# import m1.rule1 into m2 as local rule m1-rule1. IMPORT m1 : rule1 : m2 : m1-rule1 ; # import all non-local rules from m1 into m2 IMPORT m1 : [ RULENAMES m1 ] : m2 : [ RULENAMES m1 ] ;
EXPORT allows rule name aliasing across modules:
rule EXPORT ( module ? : rules * )
The EXPORT rule marks rules from the source_module as non-local (and thus exportable). If an element of rules does not name a rule in module, an error is issued. For example,
module X {
local rule r { ECHO X.r ; }
}
IMPORT X : r : : r ; # error - r is local in X
EXPORT X : r ;
IMPORT X : r : : r ; # OK.
rule CALLER_MODULE ( levels ? )
CALLER_MODULE returns the name of the module scope enclosing the call to its caller (if levels is supplied, it is interpreted as an integer number of additional levels of call stack to traverse to locate the module). If the scope belongs to the global module, or if no such module exists, returns the empty list. For example, the following prints "{Y} {X}":
module X {
rule get-caller { return [ CALLER_MODULE ] ; }
rule get-caller's-caller { return [ CALLER_MODULE 1 ] ; }
rule call-Y { return Y.call-X2 ; }
}
module Y {
rule call-X { return X.get-caller ; }
rule call-X2 { return X.get-caller's-caller ; }
}
callers = [ X.get-caller ] [ Y.call-X ] [ X.call-Y ] ;
ECHO {$(callers)} ;
rule DELETE_MODULE ( module ? )
DELETE_MODULE removes all of the variable bindings and
otherwise-unreferenced rules from the given module (or the global module,
if no module is supplied), and returns their memory to the system.
Note: though it won't affect rules that are currently executing
until they complete, DELETE_MODULE should be used with extreme
care because it will wipe out any others and all variable (including locals
in that module) immediately. Because of the way dynamic binding works,
variables which are shadowed by locals will not be destroyed, so the
results can be really unpredictable.
Boost Jam allows you to declare a local for loop control variable right in the loop:
x = 1 2 3 ;
y = 4 5 6 ;
for local y in $(x)
{
ECHO $(y) ; # prints "1", "2", or "3"
}
ECHO $(y) ; # prints "4 5 6"
Classic Jam supplies 1-based list indexing, and slicing on a closed (inclusive) range:
x = 1 2 3 4 5 ; ECHO $(x[3]) ; # prints "3" ECHO $(x[2-4]) ; # prints "2 3 4" ECHO $(x[2-]) ; # prints "2 3 4 5"
Boost Jam adds Python-style negative indexing to access locations relative to the end of the list.
ECHO $(x[-1]) $(x[-3]) ; # prints "5 3" ECHO $(x[-3--1]) ; # prints "3 4 5" ECHO $(x[-3-4]) ; # prints "3 4" ECHO $(x[2--2]) ; # prints "2 3 4"
Consistency with the 1-based, inclusive indexing of Classic Jam and the use of ``-'' as the range separator make this feature a bit clumsier than it would otherwise need to be, but it does work.
When invoking Windows-based tools from Cygwin it can be important to pass them true
windows-style paths. Boost.Jam supplies the :W modifier which,
under Cygwin only, turns a cygwin path into a Win32 path using the
cygwin_conv_to_win32_path function. On other platforms, the
string is unchanged.
x = /cygdrive/c/Program Files/Borland ; ECHO $(x:W) ; # prints "c:\Program Files\Borland" on Cygwin
Whenever a target is bound to a location in the filesystem, Boost Jam will look for a variable called BINDRULE (first ``on'' the target being bound, then in the global module). If non-empty, $(BINDRULE[1]) names a rule which is called with the name of the target and the path it is being bound to. The signature of the rule named by $(BINDRULE[1]) should match the following:
rule bind-rule ( target : path )
This facility is useful for correct header file scanning, since many compilers will search for #included files first in the directory containing the file doing the #include directive. $(BINDRULE) can be used to make a record of that directory.
For handling targets whose build actions are expected to fail (e.g. when testing that assertions or compile-time type checkin work properly), Boost Jam supplies a FAIL_EXPECTED rule in the same style as NOCARE, et. al. During target updating, the return code of the build actions for arguments to FAIL_EXPECTED is inverted: if it fails, building of dependent targets continues as though it succeeded. If it succeeds, dependent targets are skipped.
Perforce Jam supplied a NOCARE rule which is typically used for header files to indicate that if they are not found, the dependent targets should be built anyway. Boost Jam extends NOCARE to apply to targets with build actions: if their build actions exit with a nonzero return code, dependent targets will still be built.
rule RMOLD ( targets * )
Perforce Jam removes any target files that may exist on disk when the
rule used to build those targets fails. However, targets whose dependencies
fail to build are not removed by default. The RMOLD rule
causes its arguments to be removed if any of their dependencies fail to
build.
Note: the SUBST rule is deprecated in favor of
Perforce Jam's built-in MATCH rule, which has been rolled into
Boost.Jam.
The behavior of the SUBST rule for regular-expression matching and replacement (originally added in FTJam) has been modified:
SUBST ( source pattern replacements + )The return value is the concatenated results of applying each element of replacements in turn. For example, the following will print ``[x] (y) {z}'':
ECHO [ SUBST xyz (.)(.)(.) [$1] ($2) {$3} ] ;
A predefined global variable with two elements indicates the version number of Boost Jam. Boost Jam versions start at "03" "00". Earlier versions of Jam do not automatically define JAM_VERSION.
rule BACKTRACE ( )
Returns a list of quadruples: filename line module rulename..., describing each shallower level of the call stack. This rule can be used to generate useful diagnostic messages from Jam rules.
The -d command-line option admits new arguments:
Classic jam treats any non-option element of command line as a name of target to be updated. This prevented more sophisticated handling of command line. This is now enabled again but with additional changes to the UPDATE> rule to allow for the flexibility of changing the list of targets to update. The UPDATE builtin rule is:
rule UPDATE ( targets * )
The rule has two effects: 1. it clears the list of targets to update, and 2. causes the specified targets to be updated. If no target was specified with the UPDATE rule, no targets will be updated. To support changing of the update list in more usefull ways, the rule also returns the targets previously in the update list. This makes it possible to add targets as such:
local previous-updates = [ UPDATE ] ; UPDATE $(previous-updates) a-new-target ;
It is sometimes desirable to disallow parallel execution of some actions. For example:
Any target can be assigned a semaphore, by setting a variable called SEMAPHORE on that target. The value of the variable is the semaphore name. It must be different from names of any declared target, but is arbitrary otherwise.
The semantic of semaphores is that in a group of targets which have the same semaphore, only one can be updated at the moment, regardless of "-j" option.
rule W32_GETREG ( path : data ? )
Defined only for win32 platform. It reads the registry of Windows. 'path' is the location of the information, and 'data' is the name of the value which we want to get. If 'data' is omitted, the default value of 'path' will be returned. The 'path' value must conform to MS key path format and must be prefixed with one of the predefined root keys. As usual,
Other predefined root keys are not supported.
Currently supported data types : 'REG_DWORD', 'REG_SZ', 'REG_EXPAND_SZ', 'REG_MULTI_SZ'. The data with 'REG_DWORD' type will be turned into a string, 'REG_MULTI_SZ' into a list of strings, and for those with 'REG_EXPAND_SZ' type environment variables in it will be replaced with their defined values. The data with 'REG_SZ' type and other unsupported types will be put into a string without modification. If it can't receive the value of the data, it just return an empty list. For example,
local PSDK-location =
[ PROFILE HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\MicrosoftSDK\\Directories : "Install Dir" ] ;
rule ISFILE ( targets * )
ISFILE marks targets as required to be files. This changes the
way jam searches for the target such that it ignores mathes for file
system items that are not file, like directories. This makes it possible to
avoid #include "exception" matching if one happens to have a
directory named exception in the header search path.
rule SHELL ( command )
SHELL executes command, and then returns the standard output of command. SHELL only works on platforms with a popen() function in the C library. On platforms without a working popen() function, SHELL is implemented as a no-op. SHELL works on Unix, MacOS X, and most Windows compilers. SHELL is a no-op on Metrowerks compilers under Windows.
.This section is derived from the official Jam documentation and from my experience using it and reading the Jambase rules. I repeat the information here mostly because it is essential to understanding and using Jam, but is not consolidated in a single place. Some of it is missing from the official documentation altogether. I hope it will be useful to anyone wishing to become familiar with Jam and the Boost build system.
· Jam ``rules'' are actually simple procedural entities. Think of them as functions. Arguments are separated by colons.
· A Jam target is an abstract entity identified by an arbitrary string. The build-in DEPENDS rule creates a link in the dependency graph between the named targets.
· Note that the documentation for the built-in INCLUDES rule is incorrect: INCLUDES targets1 : targets2 causes everything that depends on a member of targets1 to depend on all members of targets2. It does this in an odd way, by tacking targets2 onto a special tail section in the dependency list of everything in targets1. It seems to be OK to create circular dependencies this way; in fact, it appears to be the ``right thing to do'' when a single build action produces both targets1 and targets2.
· When a rule is invoked, if there are actions declared with the same name as the rule, the actions are added to the updating actions for the target identified by the rule's first argument. It is actually possible to invoke an undeclared rule if corresponding actions are declared: the rule is treated as empty.
· Targets (other than NOTFILE targets) are associated with paths in the file system through a process called binding. Binding is a process of searching for a file with the same name as the target (sans grist), based on the settings of the target-specific SEARCH and LOCATE variables.
· In addition to local and global variables, jam allows you to set a variable on a target. Target-specific variable values can usually not be read, and take effect only in the following contexts:
· The ``bound value'' of a variable is the path associated with the target named by the variable. In build actions, the first two arguments are automatically replaced with their bound values. Target-specific variables can be selectively replaced by their bound values using the bind action modifier.
· Note that the term ``binding'' as used in the Jam documentation indicates a phase of processing that includes three sub-phases: binding (yes!), update determination, and header file scanning. The repetition of the term ``binding'' can lead to some confusion. In particular, the Modifying Binding section in the Jam documentation should probably be titled ``Modifying Update Determination''.
· ``Grist'' is just a string prefix of the form <characters>. It is used in Jam to create unique target names based on simpler names. For example, the file name ``test.exe'' may be used by targets in separate subprojects, or for the debug and release variants of the ``same'' abstract target. Each distinct target bound to a file called ``test.exe'' has its own unique grist prefix. The Boost build system also takes full advantage of Jam's ability to divide strings on grist boundaries, sometimes concatenating multiple gristed elements at the beginning of a string. Grist is used instead of identifying targets with absolute paths for two reasons:
When grist is extracted from a name with $(var:G), the result includes the leading and trailing angle brackets. When grist is added to a name with $(var:G=expr), existing grist is first stripped. Then, if expr is non-empty, leading <s and trailing >s are added if necessary to form an expression of the form <expr2>; <expr2> is then prepended.
· When Jam is invoked it imports all environment variable settings into corresponding Jam variables, followed by all command-line (-s...) variable settings. Variables whose name ends in PATH, Path, or path are split into string lists on OS-specific path-list separator boundaries (e.g. ":" for UNIX and ";" for Windows). All other variables are split on space (" ") boundaries. Boost Jam modifies that behavior by allowing variables to be quoted.
· A variable whose value is an empty list or which consists entirely of empty strings has a negative logical value. Thus, for example, code like the following allows a sensible non-empty default which can easily be overridden by the user:
MESSAGE ?= starting jam... ;
if $(MESSAGE) { ECHO The message is: $(MESSAGE) ; }
If the user wants a specific message, he invokes jam with "-sMESSAGE=message text". If he wants no message, he invokes jam with -sMESSAGE= and nothing at all is printed.
· The parsing of command line options in Jam can be rather unintuitive, with regards to how other Unix programs accept options. There are two variants accepted as valid for an option:
Please also read The Jam language reference for the additional details.
Revised 18 November, 2004
Copyright 2003-2004 Rene Rivera, David Abrahams, Vladimir Prus.
Distributed under the Boost Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt or http://www.boost.org/LICENSE_1_0.txt)