[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Written by Jorrit Tyberghein, [email protected]. Last updated 30 April 2004.
This section explains, in some detail, how the engine works internally. This information is mainly for people who are interested in learning how the Crystal Space engine works internally and for people who want to contribute to the engine proper. The sources for the 3D engine can be found in `CS/plugins/engine/3d'.
Note: This discussion is specific to how the engine works in relation to the new renderer. The old renderer is still in Crystal Space at this moment but this discussion does not bear upon it.
The following sections explain the various data structures that are used in the engine. Many of these data structures are subclasses of `csObject', which means that you can use them in your game to attach other game specific objects.
At the core of everything there is the `csEngine' class which implements the publicly viewable `iEngine' interface. The engine class implements the following interfaces:
iEngine
: The main entry point for the engine.
iObject
: Allows attachment of game-specific data.
The engine class maintains the following lists of different types of objects or operations:
sectors
: Class `csSector', interface `iSector'. As explained
in other parts of this manual, a sector is simply an infinite region of space.
There is always at least one sector.
mesh_factories
: Class `csMeshFactoryWrapper', interface
`iMeshFactoryWrapper'. Mesh factory wrappers contain templates for making
objects. A factory is not visible itself but you can create mesh objects
from a factory and place them into sectors. Note that children of hierarchical
mesh factories are not in this list.
meshes
: Class `csMeshWrapper', interface `iMeshWrapper'. These
are the actual mesh objects. Mesh objects are normally placed in sectors but
every mesh will also be present in this list so that the engine can perform
maintenance on them. A mesh object that is not in any sector, but only
in this list, will not be visible. Note that children of hierarchical
mesh objects are not in this list.
collections
: Class `csCollection', interface `iCollection'. A
collection is a simple container for other objects (meshes, lights, etc.).
This can be useful for scripts.
camera_positions
: Class `csCameraPosition', interface
`iCameraPosition'. If you define start positions in your map file, then
they will be placed into this list.
regions
: Class `csRegion', interface `iRegion'. Regions
provide a way to organize your objects. Every object (mesh, factory, material,
light, sector, etc.) can be placed into a region. Note that a region is not
a geometrical concept. You can use one region to contain all materials,
and another to contain all mesh objects, for example. On the other hand, it is
probably most useful to have regions that represent geometrical areas which
you can load and unload from the engine (useful for making huge maps).
textures
: Class `csTextureWrapper', interface
`iTextureWrapper'. A texture is a processed image that can be used by the
renderer.
materials
: Class `csMaterialWrapper', interface
`iMaterialWrapper'. A material is a surface attribute for a mesh object.
A material usually uses at least one texture but it is also possible to have
materials that do not utilize textures at all.
halos
: Class `csLightHalo', interface `iHalo'. Halos and lens
flares are in this list. This list contains all halos that were visible during
frame rendering. After the entire frame has been rendered the engine uses
this list to draw all halos that were visible.
shared_variables
: Class `csSharedVariable', interface
`iSharedVariable'. The engine maintains a list of variables that can
be used for the engine sequence manager and also for controlling LOD
(level of detail).
cleanup
: This is an array that keeps track of items that the engine must
clean up (delete) when the engine itself is destroyed.
render_priorities
: This is a list of all render priorities supported by
the engine. Render priorities control in which order objects
in a sector must be rendered.
want_to_die
: This is a set of all `iMeshWrapper' objects
that want to die the next frame. The purpose of this lists is for mesh
objects to schedule their own destruction (this can be useful for an
explosion mesh object for example). The mesh object cannot destroy itself
since it would then remove its own instance and this would be problematic.
In addition, the engine also maintains the following important variables:
Strings
: Interface `iStringSet'. The string set interface is used
for converting strings to numerical identifiers. The renderer and shader
systems uses these identifiers for tagging types of items since comparison of
numerical identifiers is faster than comparison of string identifiers.
cache_mgr
: Interface `iCacheManager'. This is the default cache
manager maintained by the engine. The cache manager is responsible for storing
data that is hard to compute. Mesh objects and other plugins may use the cache
manager to store, for example, lightmaps and other stuff.
defaultRenderLoop
: Interface `iRenderLoop'. The default renderloop
controls how objects are rendered in a sector in combination with shaders.
More on that later. Note that sectors can override the default renderloop.
renderLoopManager
: Interface `csRenderLoopManager'. The renderloop
manager is responsible for creating renderloops.
render_context
: Interface `iTextureHandle'. This is used while
rendering on a texture (i.e. when using iGraphics3D::SetRenderTarget()
is in effect).
default_shader
: Interface `iShader'. This is the default shader
that is used on all materials which do not otherwise specify a shader.
top_clipper
: Class `csRenderView'. During rendering of a frame
this variable points at the top level clipper. More on how `csRenderView'
works later.
A sector is one of the most important data structures in the engine. It is a container for other objects (meshes and lights). The sector class implements the following interfaces:
iSector
: The main entry point for the sector.
iObject
: Allows attachment of game-specific data.
The sector maintains the following lists:
meshes
: Interface `iMeshWrapper'. This is a list of all mesh
objects that are visible in this sector. Note that a mesh can be in multiple
sectors at the same time (if it is crossing a portal boundary).
camera_meshes
: Interface `iMeshWrapper'. This list contains all
meshes that have the `CS_ENTITY_CAMERA' flag set. These meshes have to be
treated a bit differently (more on that later).
portal_meshes
: Interface `iMeshWrapper'. This is a set of meshes
that have portals which leave from this sector. By using this set you can
quickly find all sectors that are reachable from this sector (through portals).
lights
: Interface `iLight'. This is a list of all lights in this
sector.
visibleMeshCache
: Class `csSector::visibleMeshCacheHolder'. This
is a special list of caches for visibility. More on this later.
usedMeshLists
: Class `csRenderMeshList'. This is a list of data
structures which need to be cleaned up when the sector itself is destroyed.
More on this later.
In addition, the sector also maintains the following important variables:
dynamic_ambient_color
: Class `csColor'. This is the current
dynamic ambient color that is added to all mesh objects in this sector.
renderloop
: Interface `iRenderLoop'. The optional renderloop for
this sector. If null, then the engine's renderloop will be used.
culler
: Interface `iVisibilityCuller'. Every sector has a
visibility culler; usually an instance of either Frustvis or
Dynavis. Frustvis is the default culler.
lights::kdtree
: Class `csKDTree'. The kdtree helps to find
lights based upon some box in the sector. Every light is added to this tree
with a bbox (bounding box) that is calculated from the center of the
light and the influence radius.
A `csMeshFactoryWrapper' is a wrapper class which the engine uses to keep
track of iMeshObjectFactory
instances (these represent the real mesh
object factories), and also stores some engine-specific information.
The mesh factory wrapper class implements the following interfaces:
iMeshFactoryWrapper
: The main entry point for the factory wrapper.
iObject
: Allows attachment of game-specific data.
A mesh factory wrapper maintains the following important variables:
meshFact
: Interface `iMeshObjectFactory'. This is the actual mesh
object factory represented by this wrapper.
parent
: Interface `iMeshFactoryWrapper'. Mesh factories can be
hierarchical. If they are, then this variable will point to the parent.
Otherwise it is null.
transform
: Class `csReversibleTransform'. This transform is only
used if this factory is a child of another factory. In that case it is the
relative position of this child relative to the parent. This is used when
instantiating mesh objects from a hierarchical factory to set up the initial
transforms.
children
: Class `csMeshFactoryFactoryList'. If this is a
hierarchical mesh factory, then the children are listed here.
static_lod
: Class `csStaticLODFactoryMesh'. If this is a
hierarchical factory with static LOD (level of detail), then this field
contains more information about that.
A `csMeshWrapper' is a wrapper class for the engine which allows it to keep track of an `iMeshObject' instance (real mesh objects), and also stores some engine-specific information. The mesh wrapper class implements the following interfaces:
iMeshWrapper
: The main entry point for the mesh object wrapper.
iObject
: Allows attachment of game-specific data.
iVisibilityObject
: Used by the visibility culling plugin. Mesh wrappers
can be registered with the visibility culler that lives in a sector.
iImposter
: Not yet implemented, but will be used for an imposter system.
A mesh object wrapper maintains the following important variables:
meshobj
: Interface `iMeshObject'. This is the actual mesh object
represented by this wrapper.
Parent
and csParent
: Interface `iMeshWrapper' and class
`csMeshWrapper'. If the mesh is part of a hierarchical mesh, then these
fields will point at the parent mesh.
children
: Class `csMeshMeshList'. This is a list of children if
this is a hierarchical mesh object.
factory
: Interface `iMeshFactoryWrapper'. Points a the factory
from which this mesh originated (optional).
movable
: Class `csMovable'. This important field contains
information about where the mesh actually resides. It maintains a list of
sectors (a mesh can reside in multiple sectors at once) and a current position
and transform. It is also aware of the hierarchical nature of meshes so that
you can ask the real position and the position relative to the parent.
zbufMode
Class `csZBufMode'. The z-buffer mode to use for
rendering this object (zfill, zuse, ztest, etc.).
render_priority
: Type `int'. This is a number indicating this
mesh's render priority.
static_lod
: Class `csStaticLODMesh'. If this mesh is the parent
for a static LOD (level of detail) mesh, then this structure will contain
more information about that.
light_info
: Interface `iLightingInfo'. Some meshes implement this
interface. The engine very often requires a pointer to the mesh's
implementation of this interface. As an optimization, it therefore maintains a
pointer to that interface here in order to avoid scfQueryInterface
,
which is somewhat time-consuming.
shadow_receiver
: Interface `iShadowReceiver'. Some meshes
implement this interface. Again, as an optimization, a pointer to this
information is maintained.
portal_container
: Interface `iPortalContainer'. A portal container
is a special type of mesh which is maintained by the engine itself and which
contains portals to other sectors. This interface can also be obtained from
the mesh object via scfQueryInterface
, but it is also maintained
here as an optimization.
relevant_lights
: Interface `iLight'. This is used for dynamic
lighting calculation, and will contain a list of all lights that currently
affect this mesh.
relevant_lights_ref
: Class `csMeshWrapper::LightRef'. This is a
mirror of `relevant_lights'. It uses `csWeakRef' as a trick for
detecting if a light has been removed or changed. (Weak references are
automatically set to null as soon as the object to which the weak reference
points is deleted.)
A region is a collection of objects. It utilizes `csObject' to hold all objects that are in the region. The region class implements the following interfaces:
iRegion
: The main entry point for the region.
iObject
: Allows attachment of game-specific data.
A texture wrapper is an engine object which keeps track of the real textures from the renderers (`iTextureHandle'). The texture wrapper class implements the following interfaces:
iTextureWrapper
: The main entry point for the texture wrapper.
iObject
: Allows attachment of game-specific data.
A texture wrapper maintains the following important variables:
image
: Interface `iImage'. This is the object which contains the
original image data from disk. After preparing this texture to make it
suitable the renderer, the image is released (unless keep_image is true).
keep_image
: Type `bool'. To prevent `image' from being
cleared, set this to true. Note that this increases memory usage of a single
texture, often dramatically, so only do this if the original image data will be
needed at a later time.
handle
: Interface `iTextureHandle'. This is the texture handle as
returned by the renderer (via the iTextureManager
interface).
A material wrapper is an engine object which keeps track of the real materials from the renderers (`iMaterialHandle'). The material wrapper class implements the following interfaces:
iMaterialWrapper
: The main entry point for the material wrapper.
iObject
: Allows attachment of game-specific data.
A material wrapper maintains the following important variables:
material
: Interface `iMaterial'. This is the original
representation of the material from which the iMaterialHandle
was
generated.
matEngine
: Interface `iMaterialEngine'. This points to the
engine-specific material class (`csMaterial', which implements both
`iMaterial' and `iMaterialEngine'). The reason for the presence of
this variable is that `iMaterial' is an interface that is not specific to
the engine so it is not possible to maintain engine-specific information there
(such as a reference to an `iTextureWrapper'). Instead, such information
is maintained in the `iMaterialEngine' interface.
handle
: Interface `iMaterialHandle'. This is the actual material
handle as returned by the renderer (via the `iTextureManager' interface).
The movable class is used by the `csMeshWrapper' class to keep track of the mesh's current location. The movable class implements the following interfaces:
iMovable
: The main entry point for the movable.
A movable maintains the following important variables:
obj
: Class `csReversibleTransform'. The world-to-object transform
for the mesh represented by this movable. For this transform This
represents object space and Other represents world space.
is_identity
: Type `bool'. If the `obj' transform is an
identity transform (i.e. neither moves nor rotates the object) then it is
possible to perform various optimizations.
sectors
: Class `csMovableSectorList'. This is a list of sectors in
which the mesh object resides In many cases a mesh will only reside in only one
sector, but it is possible for a mesh to be in several sectors at once if the
mesh happens to cross a portal boundary. Note that the application is
responsible for adding a mesh to sectors. The engine will never do so.
listeners
: Interface `iMovableListener'. This is an array of
listeners (pieces of code) interested in knowing when the movable changes (i.e.
when the mesh moves).
parent
: Interface `iMovable'. If the mesh is being used in a
hierarchical mesh object, then this will point to the movable of the parent
mesh.
object
: Class `csMeshWrapper'. This is the mesh for the movable.
updatenr
: Type `long'. One way to get notified about movable
updates is to register as a listener. The listener will be called whenever the
movable is updated. Another way is to keep track of the current
`updatenr'. This number is updated every time the movable is updated.
The light class is the representation of a light. A light lives in a sector. The light class implements the following interfaces:
iLight
: The main entry point for the light.
iObject
: Allows attachment of game-specific data.
A light maintains the following important variables:
light_id
: Type `char*'. This is a 16-byte string (MD5 hash)
which is used for caching light data. It is, hopefully, unique for a given
light since it is calculated from various light settings such as the position,
radius, sector, etc.
childnode
: Class `csKDTreeChild'. Lights are represented in a
KD-tree that lives in the sector (see above). This child object is the
representation of the light in that tree.
sector
: Interface `iSector'. The current sector of the light.
center
: Class `csVector3'. The current position of the light.
color
: Class `csColor'. The current color of the light.
halo
: Class `csHalo'. If the light has a halo, then this field
contains a pointer to that structure.
dynamic_type
: Type `int'. This represents the type of the light
(dynamic, static, etc.).
attenunation
: Type `int'. This is the kind of attenunation that
the light uses.
attenuationvec
: Class `csVector3'. If the attenuation is CLQ
based, then this will be the vector.
influenceRadius
: Type `float'. This is the influence radius of the
light. Beyond this distance the light has almost no effect.
lightnr
: Type `uint32'. This number is used to detect light
changes. It will be updated whenever the light is moved or otherwise changes.
light_cb_vector
: Interface `iLightCallback'. This is another way
to get notified of light changes.
lightinginfos
: Interface `iLightingInfo'. This is a hash set with
lighting info's which this light is currently affecting. When the light
color changes (without the light moving) the light class simply notifies every
element of this set.
This is the representation of a portal container. It is the only mesh object that is implemented directly in the engine itself. A portal container is a collection of portals (i.e. every polygon in the portal container is a portal to another sector). The portal container class implements the following interfaces:
iMeshObject
: A portal container is a mesh object.
iPortalContainer
: The portal container-specific API (this
is the state interface of the mesh object).
iShadowReceiver
: A portal container can receive shadows (and pass
them along to the destination sectors).
A portal container maintains the following important variables:
portals
: Class `csPortal'. This is a list of all portals in the
portal container.
data_nr
: Type `uint32'. This number is increased with each
significant change to the portal container.
vertices
: Class `csVector3'. This is an array of vertices that are
used by the portals in this container. These vertices are in object space.
world_vertices
: Class `csVector3'. This is an array of vertices in
world space.
camera_vertices
: Class `csVector3'. This is an array of vertices
in camera space.
meshwrapper
: Class `csMeshWrapper'. A pointer back to the mesh
wrapper containing this portal container.
This is a temporary data structure that is used during rendering. It will be discussed in more detail in the section about drawing a frame. Here is a brief summary of the class. The renderview class implements the following interfaces:
iRenderView
: The main entry point for the render view.
The renderview class maintains the following important variables:
context_id
: Type `uint32'. This is a field that is incremented
with each new csRenderView
instance (and also when a new render context
is created), and is used so the engine can detect if two render contexts are
the same (comparing on pointers is not valid since render contexts are reused).
ctxt
: Class `csRenderContext'. This is the current render context.
There is one `csRenderView' instance that is used through the entire
rendering of a frame but every time a portal is traversed a new
`csRenderContext' instance is created.
original_camera
: Interface `iCamera'. This points at the original
camera which was used when starting the rendering (i.e. the top level camera).
Normally, this camera will be constant unless there is a space warping portal
(a portal that changes space; for example, a mirror). In that case a new
camera is created for the new render context.
When rendering a frame the base entry point is iEngine::Draw()
. However,
most applications do not call this function directly. Most applications will
invoke iView::Draw()
instead.
iEngine::Draw()
accepts two arguments: the camera to use, and the
2D clipping area. When this function is called then engine will perform the
following steps in the order shown:
csEngine::ControlMeshes()
is called. This function checks
all meshes that want to die (the `csEngine::want_to_die' set) and destroys
them.
iGraphics3D
methods are invoked to ensure that the clipper is
set up correctly, among other things.
iRenderLoop::Draw()
, which will be described below. The
upshot of this method invocation is the recursive drawing of everything
visible.
iGraphics3D
is reset.
iRenderLoop::Draw()
is called for every sector in the recursion. It
will first be invoked by csEngine::Draw()
, and it will also be called
again whenever portal traversal occurs (see below). The following operations
are performed by this function in the order shown:
csRenderView::SetupClipPlanes()
is invoked. This function will
set up the clipping planes for the current 2D portal boundary (2D clipper) and
the current camera (view frustum). The far plane and near clipping planes
are also added if needed.
iShaderManager::PushVariables()
performs
this operation.
csSector::PrepareDraw()
is invoked. This function will first make
sure the visibility culler plugin for the sector is present. Then, it will
set the current sector on the `csRenderView'. Next, it will call all
callbacks that are interested in hearing when this sector is traversed.
One of the most important things this function does however, is move all
meshes that have the `CS_ENTITY_CAMERA' flag set so that their center
point is at the camera position.
What actually occurs during invocation of the rendersteps depends upon how the
renderloop is set up, and upon how many rendersteps there are. What follows is
an explanation of the default case of a single render step that uses the
old-renderer compatibility shader type (this is the renderloop that is
created by iEngine::CreateDefaultRenderLoop()
). This render step is
designed to be compatible with old renderer style maps that use lightmaps
without no special shaders. It uses `CS_ZBUF_MESH' for this render step
which means that the z-buf mode of the mesh itself will be used. It also
disables Z-offset since there will only be one step. Z-offset is used to make
sure that, when using multi-texturing, the polygons end up at exactly the same
Z-distance (i.e. no z fighting occurs). The first step is always rendered
using disabled z-offset. Subsequent steps should enable it to make sure they
will render nicely on the previously rendered base. This render step also
enables portal traversal.
iRenderStep::Perform()
resides in the generic render step plugin which
is located in the source tree at `CS/plugins/engine/renderloop/stdsteps'.
This function performs the following tasks in the order shown:
iSector::GetVisibleMeshes()
)
from the sector. A later section will go into this step in more detail.
RenderMeshes()
.
RenderMeshes()
will first set the current settings for this step
(like z-offset). Then it will get the number of passes for the given
shader and render all meshes with that shader with the given pass. Note
that the shader will also add variables to the stack at this point; in
particular variables about the current object to camera transformation and
the current material.
iPortalContainer::Draw()
. More on this later, but this is the important
place where the recursion through portals occurs.
One very important function that is called from within
iRenderStep::Perform()
is iSector::GetVisibleMeshes()
. This
function is responsible for finding all meshes that are visible. To calculate
this the sector uses the visibility culler that is in every sector. At
this moment we have two possible implementations for that visibility culler:
Frustvis and Dynavis. Frustvis performs frustum culling alone.
This means that every object that is in the view frustum is considered visible.
Dynavis does frustum culling and also attempts to find out which objects are
visible and which are not by using an occlusion buffer. There is another
chapter explaining the differences a bit more and also how Dynavis works
internally. The description here is only a short synopsis.
GetVisibleMeshes()
performs the following actions in the order shown:
GetVisibleMeshes()
is called multiple times in
one frame and it is desirable to avoid duplicating the visibility culling,
since that can be expensive (especially in the case of Dynavis). Therefore, a
cache of visible mesh lists is kept, and each entry in that cache (i.e. every
list of meshes in that cache) has an associated current frame number and
context identification number. These two numbers are used to check if the
cached list can be used for the given frame and context. If there is no such
cached list then proceed to the next step. Otherwise return the cached list
that matches the frame and context identification.
iVisibilityCuller::VisTest()
.
iVisibilityCuller::VisTest()
.
iVisibilityCuller::VisTest()
internally uses a KD-tree to traverse
the objects in the sector and, for all visible objects, it invokes the
`iVisibilityCullerListener' callback that was provided as a parameter of
the VisTest()
call. More detail about the actual implementation of
VisTest()
for both Frustvis and Dynavis will be presented in later
sections.
The callback's ObjectVisible()
is a function that resides in the sector
(the culler itself is implemented in another plugin). ObjectVisible()
will mark the given object as visible by adding all `csRenderMesh'
instances as returned by the visible `iMeshObject' (through the
iMeshObject::GetRenderMeshes()
invocation) to the list of visible meshes
which is being built. In addition, it will also do something special if the
mesh is a hierarchical mesh that uses static LOD. In such a case, it will
calculate which of the children should be visible for the given distance to the
camera (and other potential factors), and will mark that child as visible too.
It is important to note that there is an important difference between normal hierarchical meshes and static LOD meshes with regards to the visibility culler. Both are implemented using hierarchical meshes but, in case of a normal hierarchical mesh, all children are registered with the visibility culler as separate meshes. This means that every mesh in a hierarchical mesh is considered a separate entity for the visibility culler, and is also tested for visibility on its own (so a child of a hierarchical mesh can be visible even if the parent is not visible). For a static LOD mesh the situation is completely different. In that case, the children are not registered with the visibility culler but, instead, the mesh object will mark the correct child as visible when it, itself, is visible. This means that you must always make sure that, with static LOD, the parent is a mesh that is at least as big as the biggest child. Usually, a `nullmesh' with an appropriate box is used for this purpose.
While performing the renderstep, it will try traverse portals if that
renderstep has portal traversal enabled. This will proceed to
iPortalContainer::Draw()
, which will perform the following actions in
the order shown:
Prepare()
is invoked in order to ensure that the portal container is
ready for use. This function will initialize the portal container, compress
the vertices (remove double vertices), and calculate the bounding box and
sphere.
WorldToCamera()
is invoked. This will take the world space vertices and
planes and transform them to camera space.
GetRenderMeshes()
that occurred earlier (when the
renderloop called iSector->GetVisibleMeshes()
) it tested if the portal
container object needed to be clipped to the frustum or near plane. The result
of that clip test was stored in the variables clip_plane
,
clip_portal
, and clip_z_plane
. If clipping is required then, at
ths stage, it will loop over every portal and clip it before traversal. If no
clipping is required, it will simply loop over all portals and traverse them.
DrawOnePortal()
. This function
will first set up various things for fog and floating portals (like calling
iGraphics3D::OpenPortal()
, for instance), and then call
csPortal::Draw()
.
csPortal::Draw()
will first make sure the sector is present by calling
CompleteSector()
. This function invokes missing sector callbacks if the
sector is missing and if such callbacks have been registered. This gives the
application a chance to load the destination sector. If this fails then portal
traversal will not occur.
csPortal::Draw()
will also make sure portal recursion is not occurring
too deeply the same sector. This is done to avoid infinite loops caused by
space warping portals that warp back in space so that the same portal is
visible again.
@@@TODO: explain Frustvis::VisTest()
@@@TODO: explain Dynavis::VisTest()
@@@TODO: explain the operations of one mesh object in the above context
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] |
This document was generated using texi2html 1.76.