This will guide you through loading NASA's Blue Marble image mosaic data.
The data we will be using is in the ECW format.
PART 1: Getting Started
In this tutorial we will download and process coverage data and serve it up in GeoServer. The data is NASA's Blue Marble and it is in the ECW image format. You can read about the dataset here on GeoTorrents.
Step 1: Get the latest GeoServer with WCS support (GeoServer 1.5 and up)
Download the latest version of GeoServer 1.5 and above.
Step 2: Download the data
Download the image files here.
The data for this tutorial is 500m/pixel. So it is very detailed and quite large.
The data is distributed over bit torrent so make sure you have a bit torrent client such as Azureus. You're looking at a 2GB download, so start it soon.
Once the data has finished downloading, take a look at the files. You should see 12 files, 11 of them being .ecw files. Each .ecw is a different month of the year in 2004. For this example I am using August (2004,08): world-topo-bathy-200408-3x86400x43200.ecw
Step 3: Download and Install GDAL
GDAL is the Geospatial Data Abstraction Library. It does data translations and is very useful.
For this example I used the FWTools binary release for Windows.
Once FWTools has downloaded, install it.
Step 4: Process the Data
In this step we are going to translate the ECW dataset into a GeoTIFF, assuming you're using the Windows version of GDAL. The commands are not different on other operation systems, but you may not have FWTools (on Linux you can get GDAL alone without the rest of the tools).
Take the file world-topo-bathy-200408-3x86400x43200.ecw and copy it to your FWTools install directory.
Next, on windows machines, run the FWTools Shell in your Start -> All Programs -> FW Tools 1.1.3 list. A command promt window should pop up. In that window enter the following command to extract a geotiff covering North America:
gdal_translate -of GTiff -projwin -180 90 -50 -10 world-topo-bathy-200408-3x86400x43200.ecw north_america_topo_bathy.tiff
The name world_topo_bathy means the whole world with topography shading and bathymetry shading.
The bounding box parameters (xUpperLeft, yUpperLeft, xBottomRight, yBottomRight) is around north america. If you try to do the whole world you will hit the 4GB limit for TIFFs and GDAL won't compute it.
Once the script has finished executing (it might take a good 10 minutes) you will have a large file called north_america_topo_bathy.tiff
Step 5: Load Into GeoServer
1. Copy your TIFF file to GeoServer's data directory. I put mine under [data_dir]/coverages/bluemarble/
2. Start up GeoServer
3. Go to Welcome | Config | Data | CoverageDataSets | New via the web interface, choose "Tagged Image File Format with Geographic Information" for "Coverage Data Set Description" and give it a name. I called it bluemarble.
4. Next head to Welcome | Config | Data | Coverage | New and select your Coverage Data Set (bluemarble) from the drop down list and hit the New button.
5. Set the style of your coverage to be Raster, then hit Submit and then Apply.
6. Take a look at your coverage in the map preview section of geoserver Welcome | Demo | Map Preview
The image is quite large so may take a few moments to render.
If you want to get better performance out of it, we have a tutorial dedicated to high performance coverage serving. Just follow the next steps:
PART 2: Use A Pyramid For The Best Performance
 | Warning Building pyramids is at the moment not for the faint of heart. Tools are still unstable, so be prepared to setup a developer environment and hack your way to pyramid construction. |
Now that you have a decent feeling for how this process works, lets get all of the blue marble data into GeoServer in such a way that it will be speedy to serve up.
We will be using the same dataset world-topo-bathy-200408-3x86400x43200.ecw and will produce several pyramids out of it.
But first lets get some background information about how Pyramids work.
A pyramid is several layers of the original image that have been shrunk down into smaller and smaller steps, each one to be rendered at different zoom levels. So, when you start zoomed out, you will view the smaller version of the original image. As you zoom in you will start to view versions of the original image that are getting larger and closer to the original image size.
Below is an example of how it might look. It lists each level, the scale of the original image that each level is, and the size of the image at that level:
| Level | Scale | Image Size | Zoom |
|---|---|---|---|
| 7 | 1/64 | 312.5 kB | Zoomed out |
| 6 | 1/32 | 625 kB | |
| 5 | 1/16 | 1,250 kB | |
| 4 | 1/8 | 2,500 kB | Mid-way |
| 3 | 1/4 | 5,000 kB | |
| 2 | 1/2 | 10,000 kB | |
| 1 | 1 | 20,000 kB | Zoomed right in |
Above, each layer is half the size of the previous layer; the starting image is the 20,000 kB image.
Having the image divided into levels like that helps performance a lot, but only if we are zoomed out. As soon as we start zooming in we will start hitting the large original image (20,000 kB one). In order to speed up that layer, and some of the layers in between, we need to chop them up and create a mosaic out of them. This is called tiling. How this works is that only the tiles that are currently in the bounding box will be rendered.
In the end, the pyramid will contain several layers of image mosaics
Step 1: Have coverage tools at hand
You can check out the source code and project for the coverage tools with SVN here: *http://svn.geotools.org/geotools/branches/2.3.x/*
The coverage tools can be found under: /geotools/branches/2.3.x/ext/coverage_development
Make sure you have Maven installed, there are instructions on the developers guide for GeoServer for how to do that.
With maven set up navigate to the root of the geotools source you just checked out and type:
mvn clean install
Next, navigate to geotools/branches/2.3.x/ext/coverage_development and type:
mvn clean install
Then type:
mvn eclipse:eclipse
This will create an eclipse project you so can run the tools form the IDE.
Next, start up Eclipse and import the project from the geotools/branches/2.3.x/ext/coverage_development directory.
Step 1: Translate and Chop up the Data
If we were to translate the whole dataset into one TIFF, it would be too large for the TIFF file format. So we need to chop it up into smaller TIFFs.
We will use GDAL again to do this, similar to what we did above but this time we will change the bounds of images. In order to produce managable file sizes, we are going to chop up the world into 8 pieces:
| 01 | 11 | 21 | 31 |
| 00 | 10 | 20 | 30 |
You can probably get away with chopping it up into 4 pieces, but I haven't had it work successfully when building the pyramid.
Enter the following commands into GDAL through FWTools:
gdal_translate -of GTiff -co "TILED=YES" -projwin -180 0 -90 -90 world-topo-bathy-200408-3x86400x43200.ecw bluemarble_00.tiff gdal_translate -of GTiff -co "TILED=YES" -projwin -180 90 -90 0 world-topo-bathy-200408-3x86400x43200.ecw bluemarble_01.tiff gdal_translate -of GTiff -co "TILED=YES" -projwin -90 0 0 -90 world-topo-bathy-200408-3x86400x43200.ecw bluemarble_10.tiff gdal_translate -of GTiff -co "TILED=YES" -projwin -90 90 0 0 world-topo-bathy-200408-3x86400x43200.ecw bluemarble_11.tiff gdal_translate -of GTiff -co "TILED=YES" -projwin 0 0 90 -90 world-topo-bathy-200408-3x86400x43200.ecw bluemarble_20.tiff gdal_translate -of GTiff -co "TILED=YES" -projwin 0 90 90 0 world-topo-bathy-200408-3x86400x43200.ecw bluemarble_21.tiff gdal_translate -of GTiff -co "TILED=YES" -projwin 90 0 180 -90 world-topo-bathy-200408-3x86400x43200.ecw bluemarble_30.tiff gdal_translate -of GTiff -co "TILED=YES" -projwin 90 90 180 0 world-topo-bathy-200408-3x86400x43200.ecw bluemarble_31.tiff
Note that you can substitute the relative path names with absolute ones.
Now you should have the following files:
bluemarble_00.tiff bluemarble_01.tiff bluemarble_10.tiff bluemarble_11.tiff bluemarble_20.tiff bluemarble_21.tiff bluemarble_30.tiff bluemarble_31.tiff
Step 2: Build the Mosaic
Files as is cannot be used directly into the pyramid plugin, we should first mosaic them.
First of all, we'll move them into a separate directory, to avoid confusing the mosaic index builder:
mkdir mosaic mv bluemarble_??.tiff mosaic
Then we can build a mosaic using:
MosaicIndexBuilder /path/to/the/mosaic/folder
This will generate a shapefile and a property file that do describe the mosaic.
Step 3: Build the Pyramid
All right, time to build the pyramid. To do this by hand with GDAL is tedious. You essentially have to scale the original image into your layers, then chop up each layer, and then build an image mosaic out of each layer, then finally group them together into a pyramid. Luckily we built a tool to do all this for you.
Create a a new Run... task with the following parameters:
Main Class: it.geosolutions.utils.imagepyramid.PyramidBuilder
Arguments:
Program Arguments: -s path/to/mosaic/index.shp -f 2 -n 4 -t "1024,1024" -w
VM Arguments: -Xmx512m
-s is the path to your mosaic file that will be chopped up.
-f is the scale factor to shrink the layer by for each level.
-n is the number of levels to generate. For example, 2 will generate directories 0,2,4; 4 will generate 0,2,4,8,16
-t the dimentions of the individual tiles that the original image will be chopped up into
-w leave it blank
The path to the file can be the full path to the image file.
Once the parameters are set up, run the program and it will generate your pyramid, hopefully without errors.
Step 3: Loading The Pyramids Into GeoServer
Our image mosaic is now ready to load into GeoServer, so lets go ahead and do that.
1) Start up GeoServer, then navigate to Config->Data->CoverageStores->New
2) Select Image Pyramidal Plugin then enter a name for your pyramid. Hit the New button.
3) You will now be at the CoverageStore edit screen. Enter in the location of the pyramid. If it is in the data directory, then it can be referenced relatively to it. If it is somewhere else on your file system, you can give it the absolute path.
The file you want to point at is the pyramid.properties file at the root of the pyramid structure that was generated.
When finished, hit the Submit button.
4) Navigate to the Config->Data->Coverages-New menu:
5) Select your coverage store from the drop down list and hit New.
6) You are now at the Coverage edit screen. Simply scroll to the bottom and hit Submit, then Apply and Save. You will now be able to view the Pyramid through GeoServer.
Improvements
It is possible to transform this into a single image pyramid, but it will take a bit of work. Of course it would be easier to just have to deal with one coverage, instead of the 8. Theoritically. One thing you can do is create a layer group that contains all of the pyramids.
Posted by timrobertson100 at Mar 15, 2007 09:07
