Different scanning algorithm are encapsulated by objects called "scanners". Please see the documentation for "scanner" module for more details.

As said above, it's possible to compile a C++ file twice, with different include paths. Therefore, include dependencies for those compilations can be different. The problem is that bjam does not allow several scans of the same target.

The solution in Boost.Build is straigtforward. When a virtual target is converted to bjam target (via virtual-target.actualize method), we specify the scanner object to be used. The actualize method will create different bjam targets for different scanners.

All targets with specific scanner are made dependent on target without scanner, which target is always created. This is done in case the target is updated. The updating action will be associated with target without scanner, but if sources for that action are touched, all targets — with scanner and without should be considered outdated.

For example, assume that "a.cpp" is compiled by two compilers with different include path. It's also copied into some install location. In turn, it's produced from "a.verbatim". The dependency graph will look like:

a.o (<toolset>gcc)  <--(compile)-- a.cpp (scanner1) ----+
a.o (<toolset>msvc) <--(compile)-- a.cpp (scanner2) ----|
a.cpp (installed copy)    <--(copy) ----------------------- a.cpp (no scanner)
                                                                 ^
                                                                 |
                       a.verbose --------------------------------+

This requirement breaks down to the following ones.

The first requirement means that when determining what "a.h" means, when found in "a.cpp", we have to iterate over all directories in include paths, checking for each one:

Classic Jam has built-in facilities for point (1) above, but that's not enough. It's hard to implement the right semantic without builtin support. For example, we could try to check if there's targer called "a.h" somewhere in dependency graph, and add a dependency to it. The problem is that without search in include path, the semantic may be incorrect. For example, one can have an action which generated some "dummy" header, for system which don't have the native one. Naturally, we don't want to depend on that generated header on platforms where native one is included.

There are two design choices for builtin support. Suppose we have files a.cpp and b.cpp, and each one includes header.h, generated by some action. Dependency graph created by classic jam would look like:

a.cpp -----> <scanner1>header.h  [search path: d1, d2, d3]


                  <d2>header.h  --------> header.y
                  [generated in d2]
           
b.cpp -----> <scanner2>header.h [ search path: d1, d2, d4]

In this case, Jam thinks all header.h target are not realated. The right dependency graph might be:

a.cpp ---- 
          \
           \     
            >---->  <d2>header.h  --------> header.y
           /       [generated in d2]
          / 
b.cpp ----

or

a.cpp -----> <scanner1>header.h  [search path: d1, d2, d3]
                          |
                       (includes)
                          V
                  <d2>header.h  --------> header.y
                  [generated in d2]
                          ^
                      (includes)  
                          |
b.cpp -----> <scanner2>header.h [ search path: d1, d2, d4]

The first alternative was used for some time. The problem however is: what include paths should be used when scanning header.h? The second alternative was suggested by Matt Armstrong. It has similiar effect: add targets which depend on <scanner1>header.h will also depend on <d2>header.h. But now we have two different target with two different scanners, and those targets can be scanned independently. The problem of first alternative is avoided, so the second alternative is implemented now.

The second sub-requirements is that targets generated to "bin" directory are handled as well. Boost.Build implements semi-automatic approach. When compiling C++ files the process is:

After this is done, dependencies are found by the approach explained previously.

Note that if a target uses generated headers from other main target, that main target should be explicitly specified as dependency property. It would be better to lift this requirement, but it seems not very problematic in practice.

For target types other than C++, adding of include paths must be implemented anew.

Suppose file "a.cpp" includes "a.h" and both are generated by some action. Note that classic jam has two stages. In first stage dependency graph graph is build and actions which should be run are determined. In second stage the actions are executed. Initially, neither file exists, so the include is not found. As the result, jam might attempt to compile a.cpp before creating a.h, and compilation will fail.

The solution in Boost.Jam is to perform additional dependency scans after targets are updated. This break separation between build stages in jam — which some people consider a good thing — but I'm not aware of any better solution.

In order to understand the rest of this section, you better read some details about jam dependency scanning, available at this link.

Whenever a target is updated, Boost.Jam rescans it for includes. Consider this graph, created before any actions are run.

A -------> C ----> C.pro
     /
B --/         C-includes   ---> D

Both A and B have dependency on C and C-includes (the latter dependency is not shown). Say during building we've tried to create A, then tried to create C and successfully created C.

In that case, the set of includes in C might well have changed. We do not bother to detect precisely which includes were added or removed. Instead we create another internal node C-includes-2. Then we determine what actions should be run to update the target. In fact this mean that we perform logic of first stage while already executing stage.

After actions for C-includes-2 are determined, we add C-includes-2 to the list of A's dependents, and stage 2 proceeds as usual. Unfortunately, we can't do the same with target B, since when it's not visited, C target does not know B depends on it. So, we add a flag to C which tells and it was rescanned. When visiting B target, the flag is notices and C-includes-2 will be added to the list of B's dependencies.

Note also that internal nodes are sometimes updated too. Consider this dependency graph:

a.o ---> a.cpp
            a.cpp-includes -->  a.h (scanned)
                                   a.h-includes ------> a.h (generated)
                                                                 |
                                                                 |
            a.pro <-------------------------------------------+

Here, out handling of generated headers come into play. Say that a.h exists but is out of date with respect to "a.pro", then "a.h (generated)" and "a.h-includes" will be marking for updating, but "a.h (scanned)" won't be marked. We have to rescan "a.h" file after it's created, but since "a.h (generated)" has no scanner associated with it, it's only possible to rescan "a.h" after "a.h-includes" target was updated.

Tbe above consideration lead to decision that we'll rescan a target whenever it's updated, no matter if this target is internal or not.

debug/optimization-off
debug/main-target-a