| Author: | Jeff McKenna |
|---|---|
| Contact: | jeffmckenna at gmail.com |
| Revision: | $Revision: 8365 $ |
| Date: | $Date: 2008-12-31 07:49:02 -0800 (Wed, 31 Dec 2008) $ |
| Last Updated: | 2008/06/20 |
Table of Contents
Starting with version 3.5, MapServer included the ability to access vector data sets in formats other than Shapefile in their native format using the OGR library. The following document describes the process for implementing OGR support within MapServer applications.
Note
Experimental OGR support was included in MapServer version 3.4 but this initial implementation had some limitations and is not covered in this document.
This document assumes that you are already familiar with certain aspects of MapServer:
Readers should also check out the Vector Data Access Guide, which has lots of examples of how to access specific vector formats.
The OGR Simple Features Library is a C++ open source library (and command-line tools) providing read (and sometimes write) access to a variety of vector file formats including ESRI Shapefiles, and MapInfo mid/mif and TAB formats.
OGR is actually part of the GDAL library, so you will notice that some references point to GDAL (such as the mailing list).
The OGR Simple Features Library allows MapServer users to display several types of vector data files in their native formats. For example, MapInfo Mid/Mif and TAB data do not need to be converted to ESRI shapefiles when using OGR support with MapServer.
See http://www.gdal.org/ogr/ogr_formats.html for the latest list of supported formats. At the date this document was written, the following formats were supported:
Note
Some of the above formats (e.g. OGDI) have external dependencies and are not always included in the pre-compiled binary distributions of MapServer with OGR support.*
Note
Some of the above formats are not well suited for random access by nature, that’s the case of MapInfo MIF/MID files which is a TEXT format and will give very poor performance for a web application. On the other hand, some binary formats such as MapInfo TAB are better suited for random access and will give performance comparable to native shapefile access in MapServer.*
The main developer of the OGR library is Frank Warmerdam and the integration of OGR within MapServer was done by Daniel Morissette.
For UNIX users, see the README.CONFIGURE file in the MapServer source, or see the UNIX Compilation and Installation. If GDAL/OGR is normally installed it should be sufficient to add –with-ogr to the configure line before (re)building MapServer. Linux users might want to try FGS, a Linux installer for MapServer.
For Windows users, it is recommended to look for a pre-compiled binary on the MapServer site (MS4W is recommended). If you want to compile your own then see the README.WIN32 file in the MapServer source.
The only change that is needed to integrate OGR support with a MapServer application is with the .map file. The LAYER’s DATA parameter is expanded to three parameters (CONNECTIONTYPE OGR, CONNECTION and DATA).
The syntax for this differs depending on the type of data being used (the Vector Data Access Guide is an excellent resource for this). In OGR, a data source can be either a set of files that share a common basename (e.g. .shp/.shx/.dbf for ArcView Shapefiles, or .tab/.map/.dat/.ind/.id for MapInfo TAB files) or a whole directory of files (e.g. TIGER).
Let’s call the former “File-based data sources” and the later “Directory-based data sources”. When accessing a file-based data source you specify the filename of one of the files in the set (e.g. roads.shp or roads.tab) and when accessing a directory-based data source you specify the directory name and OGR reads all the files in the directory as a single data source with potentially several layers (e.g. TIGER files).
Some OGR drivers (e.g. SHP, TAB) can have dual behaviors, that is if they’re pointed to a single file then they behave as a file-based data source and if they’re pointed to a directory then they will behave as a directory-based data source and then every file in the directory becomes a new layer in the data source.
See the OGR formats page for more info on the specific file format you’re using. (Click on the format name for more specific driver info on that format)
The .map file LAYER definition for file-based sources is as follows:
LAYER
...
CONNECTIONTYPE OGR
CONNECTION "<datasource_name>"
DATA "<layer_definition>"
...
END
<datasource_name> is the name of the datasource to read from and is prefixed by the CONNECTION keyword. The exact organization depends on the format driver in use. The format driver to use is automatically selected by OGR based on the nature of the string passed as the datasource, and/or the format of the file referenced by it.
For file based datasources this is the name of the file, including the extension, using an absolute path, or a relative path. Relative paths are interpreted relative to the SHAPEPATH first, if not found then we try again relative to the .map file location.
Note
Before version 4.1 the SHAPEPATH was ignored for OGR datasources.
For directory based datasources, such as TIGER/Line, or Arc/Info Binary Coverages this is the name of the directory containing the files. If the path is relative it is interpreted relative to the .map file.
For virtual datasources such as database systems, and OGDI this is the service connection string and is generally not related to the filesystem. For instance, for Oracle Spatial this might be “OCI:warmerda/Password@gdal800.velocet.ca”.
<layer_definition> is the name, number or SQL definition of the layer to use from the datasource. It is indicated via the DATA keyword in the map file.
- Layer Name: The (case insenstive) layer name may be used to select a layer.
- Layer Number: The layer number (starting from 0 for the first layer) may be used to select a layer. Generally the layer name is preferred to this since it is more self describing.
- Omitted: If no DATA keyword is provided, this is equivalent to selecting layer 0.
- SQL SELECT: If an SQL SELECT statement is used, it is interpreted in a driver specific manner to try and generate a temporary pseudo-layer. For some formats this a restricted subset of SQL is interpreted within OGR. For RDBMS based drivers (such as PostGIS and Oracle) this is passed through to the underlying database.
The OGRINFO utility can be used to find out the list of layers and their names in a data source.
Please see the Vector Data Access Guide for details and examples of each data format supported.
Example 1. MapInfo TAB file
LAYER
NAME "Builtup_Areas_tab"
TYPE POLYGON
CONNECTIONTYPE OGR
CONNECTION "data/tab/092b06_builtup_a.tab"
STATUS ON
CLASS
...
END
...
END
Example 2. Microstation DGN file using <layer_index>
The entire DGN file is represented in OGR as one layer (see the DGN driver page for more details):
LAYER
NAME "dgn"
TYPE LINE
CONNECTIONTYPE OGR
CONNECTION "dgn/santabarbara02.dgn"
DATA "0"
STATUS ON
STYLEITEM "AUTO"
CLASS
END
END # Layer
Example 3. TIGER/Line file using <layer_name>
LAYER
NAME "Roads_tig"
TYPE line
CONNECTIONTYPE OGR
CONNECTION "full/path/to/tiger/TGR25001"
DATA "CompleteChain"
STATUS ON
CLASS
...
END
END
Example 4. Directory of Shapefiles using SQL JOIN
LAYER
NAME "Parks_cov"
TYPE POLYGON
CONNECTIONTYPE OGR
CONNECTION "data/shppoly"
DATA "SELECT eas_id, idlink.Name FROM poly LEFT JOIN idlink ON poly.eas_id = idlink.eas_id"
STATUS ON
CLASSITEM "idlink.Name"
CLASS
...
END
END
OGRINFO is part of the GDAL/OGR distribution (it is also included in FWTools and MS4W). It is an executable that can be used to obtain layer information about OGR supported files. The parameters are:
ogrinfo [-ro] [-q] datasource_name [layer [layer...]]
- -ro opens the file as read only (optional)
- -q executes in quiet mode, only the layer idex line will be returned (optional)
- datasource_name is the filename including extension (eg. roads.tab); for TIGER/Line files, datasource_name is the directory containing the TIGER files (eg. ogrinfo TGR25001)
Example 5. To get the list of layers in a file:
$ ogrinfo popplace.tab
Had to open data source read-only.
INFO: Open of `popplace.tab'
using driver `MapInfo File' successful.
1: popplace (Point)
which shows that there is one point layer in the popplace.tab file.
Example 6. To get a dump of a specific layer, including field names, projection, etc:
$ ogrinfo popplace.tab popplace
Had to open data source read-only.
INFO: Open of `popplace.tab'
using driver `MapInfo File' successful.
Layer name: popplace
Geometry: Point
Feature Count: 497
Layer SRS WKT: PROJCS["unnamed",GEOGCS["unnamed",DATUM["North ...snipped...
AREA: Real (15.3)
PERIMETER: Real (15.3)
POPPLACE_: Real (11.0)
POPPLACE_I: Real (15.0)
NAME: String (50.0)
OGRFeature(popplace):1
AREA (Real) = 0.000
PERIMETER (Real) = 0.000
POPPLACE_ (Real) = 1
POPPLACE_I (Real) = 1
NAME (String) = Port Hope Simpson
POINT (2437287.249 1153656.751)
OGRFeature(popplace):2
AREA (Real) = 0.000
PERIMETER (Real) = 0.000
POPPLACE_ (Real) = 2
POPPLACE_I (Real) = 1
NAME (String) = Hopedale
...
...
Example 7. To get a list of layers in a TIGER/Line Directory:
$ ogrinfo TGR25001
Had to open data source read-only.
INFO: Open of `TGR25001'
using driver `TIGER' successful.
1: CompleteChain (Line String)
2: AltName (None)
3: FeatureIds (None)
4: ZipCodes (None)
5: Landmarks (Point)
6: AreaLandmarks (None)
7: KeyFeatures (None)
8: Polygon (None)
9: EntityNames (Point)
10: IDHistory (None)
11: PolyChainLink (None)
12: PIP (Point)
13: TLIDRange (None)
14: ZipPlus4 (None)
The above example shows that there are 14 layers in the TGR25001 directory.
Example 8. To get a summary of a specific TIGER layer, including only field names, projection, and extent
$ ogrinfo TGR25001 Landmarks -summary
Had to open data source read-only.
INFO: Open of `TGR25001'
using driver `TIGER' successful.
Layer name: Landmarks
Geometry: Point
Feature Count: 777
Extent: (-70.674324, 41.519817) - (-69.969211, 42.046868)
Layer SRS WKT: GEOGCS["NAD83",DATUM["North_American_Datum_1983",
SPHEROID["GRS 1980",6378137,298.257222101]],PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433]]
MODULE: String (8.0)
FILE: String (5.0)
STATE: Integer (2.0)
COUNTY: Integer (3.0)
LAND: Integer (10.0)
SOURCE: String (1.0)
CFCC: String (3.0)
LANAME: String (30.0)
OGR layers can be queried the same way as regular shapefiles in MapServer.
OGR layers can utilize tile indexes in a similar fashion to Shapefile based layers. The TILEINDEX keyword should contain the connection string for the tile index file. The tile index file may be any supported OGR format, including shapefiles.
The TILEITEM keyword in the LAYER definition indicates what attribute from the tile index file should be used as the datasource location. If omitted, the default TILEITEM value is “location”. The value in the location field should be a connection string the same as would have been used in the CONNECTION field for OGR layers. The CONNECTION keyword is not needed (and will be ignored) for layers using the OGR connection type and having the TILEINDEX keyword.
Tileindex files can be prepared in an external GIS, or using the OGR utility ogrtindex. Details can be found on the OGR Utilities Page.
The following is a simple example of a point layer using a tile index.
LAYER
NAME "ogr_points"
TYPE POINT
CONNECTIONTYPE OGR
TILEINDEX "PIP_ogr_tiles.shp,0"
STATUS ON
CLASS
SYMBOL "default-circle"
COLOR 255 0 0
SIZE 6
END
END
OGR tileindex layers should support all normal query and attribute fetching mechanisms, including from MapScript; however, this has not been heavily tested as of April/2002. Please report problems via the MapServer Trac. If auto projection support is used for tileindexed OGR layers, the tileindex is read for the projection (not the component tiles). Problems may (or may not) be encountered if the component tiles have differing schemas (different sets of attributes).
For some OGR supported formats, connecting to the dataset is quite expensive in terms of CPU use and amount of disk IO. For instance, establishing access to an S-57 dataset results in a complete read into memory of the data files. Connection pooling control aims at reducing this overhead in situations where the same file is used for several different map layers.
To ensure that an OGR supported dataset is only opened once per map render (instead of separately for each map LAYER referencing the dataset, use the CLOSE_CONNECTION PROCESSING option. The default value is for CLOSE_CONNECTION is NORMAL, but if set to DEFER the dataset will be kept open till the map render is complete. It will be reused by any other layers with using the same datasource.
Example 9. Preserve S-57 connection for two layers
In this example, we are using the same dataset (NO410810.000) for two layers. To avoid re-reading the dataset, we mark the first layer to defer closing the connection till layer. In the second (or last) layer we request NORMAL connection handling (though this could have been left out as normal handling is the default).
LAYER
NAME "AdminAreas"
TYPE POLYGON
CONNECTIONTYPE OGR
CONNECTION "NO410810.000"
DATA "ADMARE"
PROCESSING "CLOSE_CONNECTION=DEFER"
STATUS ON
...
END
LAYER
NAME "Land Areas"
TYPE POLYGON
CONNECTIONTYPE OGR
CONNECTION "NO410810.000"
DATA "LNDARE"
PROCESSING "CLOSE_CONNECTION=NORMAL"
STATUS ON
...
END
Note
This feature is only supported with MapInfo TAB and Microstation DGN files at the moment, but eventually other formats that carry colors and styles at the shape-level may also be supported through OGR.*
In MapServer, ArcView, and other shapefile-based applications, colors and styles are usually defined at the layer level. This means that all the shapes in a given layer are usually rendered using the same color and styles.
On the other hand, some formats supported by OGR such as MapInfo TAB do have color and style information attached to each shape. OGR adds support for the ‘STYLEITEM “AUTO”’ layer parameter which allows you to request that the shapes in a layer be rendered using colors and styles coming from the data source instead of being driven by CLASSes as was traditionally done with MapServer.
In order to have a layer rendered using colours and styles coming from the OGR data source, your must do the following:
Example 10. Layer Definition Using STYLEITEM “AUTO” without a CLASSITEM
LAYER
NAME "test_dgn"
STATUS ON
TYPE POLYGON
CONNECTIONTYPE OGR
CONNECTION "../data/dgn/test.dgn"
# This enables use of colors and styles from the source file.
STYLEITEM "AUTO"
# Define an empty class that will be filled at runtime from the color and
# styles read on each shape in the source file.
CLASS
END
END # layer
Example 11. Layer Definition Using STYLEITEM “AUTO” with a CLASSITEM
LAYER
NAME "Builtup_Areas_tab"
TYPE POLYGON
CONNECTIONTYPE OGR
CONNECTION "data/tab/092b06_builtup_a.tab"
STATUS ON
# This enables use of colors and styles from the source file.
STYLEITEM "AUTO"
# Define an empty class that will be filled at runtime from the color and
# styles read on each shape in the source file.
CLASSITEM "CATEGORY"
CLASS
EXPRESSION "1"
END
END
Please Note:
CLASS EXPRESSIONs are still working, so it is still possible to query and classify layers that are using STYLEITEM “AUTO”. The only difference is that instead of using static class definitions, the colors and style will be read from the data file.
| NOTE 1: | Even though MapInfo and other OGR data sources may support layers with mixed geometry types (e.g. points, lines and polygons in the same file) this is not yet supported in MapServer. So you still have to define a layer ‘TYPE’ and make sure that all the shapes in the OGR data source are compatible with that layer type, otherwise MapServer may produce an error about incompatible geometry types at runtime. |
|---|---|
| NOTE 2: | Due to the dynamic nature of this feature, it is not compatible with the labelcache, so the labelcache is automatically disabled for layers that make use of ‘STYLEITEM “AUTO”’. |
| NOTE 3: | When you use STYLEITEM AUTO, MapServer tries to match symbol names returned by OGR to names in your symbol file. For a quick solution, try using the following symbol file: |
http://www2.dmsolutions.ca/msapps/yk_demo/etc/symbols_mapinfo.sym
The name of the symbols returned by OGR to MapServer depends on the file format. In the case of MapInfo files, it will be:
The OGRINFO utility can be used to find out exactly which symbol names OGR will return to MapServer. Look at the “Style” string in the ogrinfo output for each shape that is read.
Here is the list of style parameters that are currently supported from OGR data sources and how they’re mapped in MapServer:
The default will be 1 pixel line (as it always is with MapServer). In MapServer, in order to get lines thicker than 1 pixel, one has to define a circle symbol that will be used to render the line (by applying a size to the circle symbol).
So if your data source contains lines thicker than 1 pixel then you need to provide a circle symbol in your symbolset and you must name it “default-circle”. If this symbol is present then the lines will be drawn using their real thickness, otherwise all lines will be 1 pixel wide.
Polygon fill color is mapped directly to CLASS.COLOR
Note that at this time, transparent polygons are not supported (they’re always opaque).
Point symbol color is directly mapped to CLASS.COLOR Point symbol size is directly mapped to CLASS.SIZE
If your symbolset contains a symbol called “default-marker” then this symbol will be used, otherwise the default will be CLASS.SYMBOL=0 (i.e. a 1 pixel dot)
It is also possible (with a bit of work) to control which symbol gets used in rendering point symbols. OGR provides MapServer with symbol names, and if the symbol name returned by OGR to MapServer matches the name of one of the symbols in your symbolset then this symbol will be used.
For MapInfo point symbols (numbered 32 to 67 in the MapInfo MIF spec), the name returned by OGR is “mapinfo-sym-X” where X should be replaced with the MapInfo symbol number (e.g. “mapinfo-sym-35” is the star symbol).
The text string is mapped to CLASS.TEXT
Text color is mapped to CLASS.LABEL.COLOR
Text background color is mapped to CLASS.LABEL.BACKGROUNDCOLOR
Text height is mapped to CLASS.LABEL.SIZE
Text angle is mapped to CLASS.LABEL.ANGLE
Text font mapping follows the following rules:
The following sites use OGR’s STYLEITEM “AUTO” feature:
The following site uses OGR, as well as MapInfo’s ‘Seamless Map Layers’ feature:
The following site uses OGR to display TIGER 2000 files:
| Q: | What Does “OGR” Stand For? |
|---|---|
| A: | Basically, OGR does not stand for anything. For a detailed explanation of how OGR was named, see GDAL’s FAQ at http://trac.osgeo.org/gdal/wiki/FAQ. |
| Q: | When using STYLEITEM AUTO, what should I have in my .sym symbols file? |
|---|---|
| A: | When you use STYLEITEM AUTO, MapServer tries to match symbol names returned by OGR to names in your symbol file. For a quick solution, try using the following symbol file: http://www2.dmsolutions.ca/msapps/yk_demo/etc/symbols_mapinfo.sym The name of the symbols returned by OGR to MapServer depends on the file format. In the case of MapInfo files, it will be:
The OGRINFO utility can be used to find out exactly which symbol names OGR will return to MapServer. Look at the “Style” string in the ogrinfo output for each shape that is read. |