[Geoserver-users] Should I build pyramids or overviews?

GeoServer GeoServer User
1

Correction: I meant huge raster datsaset as in several Terabytes, not Gigabytes

–Steve

···

From: Stephen Brooke
Sent: Monday, July 28, 2014 5:32 PM
To: geoserver-users@lists.sourceforge.net
Subject: Should I build pyramids or overviews?

I have a huge raster dataset of GeoTiff files (several gigabytes of satellite imagery) and I am hoping to use GeoServer 2.5.1 and OpenLayers to provide a seamless pan and zoom capability for our users. I have done a lot of research and reading about how this can be done and have tested out the ImagePyramid and ImageMosaic plugins for GeoServer on a small sample dataset.

In particular I have read through much of the GeoServer docs and have also started looking at the ESRI docs because they have useful guidelines about whether to build pyramids or overviews but since this is the first time that I’m doing this it is difficult to know what approaches will turn out the best.

http://docs.geoserver.org/latest/en/user/production/data.html

http://blogs.esri.com/esri/arcgis/2012/11/14/should-i-build-pyramids-or-overviews/

So far my plan is to automate the following process each time a new GeoTiff file is received:

  • Create an image pyramid (using gdal_retile.py)

  • Create a new ImagePyramid Coverage Store and publish a new coverage layer based on this coverage store in GeoServer (using GeoBatch)

  • Somehow add metadata/attributes about the coverage layer so it can be returned in CQL query results

  • Add the new layer to the appropriate Layer Group(s) in GeoServer (based on categories)

Can anybody tell me if I am I on the right track? Are there viable alternative approaches?

–Steve

2

Hi Stephen,

Inevitably you can’t have a single image file that’s TB-size (or even 5+GB) so you’ll have thousands, and overviews with mosaicing won’t particularly help - when you render a “zoomed out” view of the layer, the server has to do a tonne of IO to access thousands of 10px x 10px overviews from each of the relevant tiles. OTOH, with a pyramid, the imagery is retiled at each “level” so the server has to read less imagery (normally a maximum of 4x tiles) to render a single response.

You might find it helpful to add overviews (eg. 2x) to pyramid images and render fewer pyramid levels (eg. 1x 4x 16x rather than 1x 2x 4x 8x 16x) - but there are a number of different aspects (disk performance, storage/size constraints, image format, compression, tile size, memory, CPU) which all interact in weird & wonderful ways to affect performance - so you’ll need to do some testing in your environment to see what works for you.

HTH, good luck!

Rob :slight_smile:

···

–

Koordinates
PO Box 1604, Shortland St, Auckland 1140, New Zealand
Phone +64-9-966 0433 koordinates.com