Hi Bart,
It’s absolutely possible to publish your 200 GB of geotiffs as a single imagemosaic layer.
We have an imagemosaic layer comprised of 531 individual geotiffs (stitched seafloor imagery mosaics collected from AUVs). Each tiff represents a single AUV deployment and ranges from a few MB to 25 GB. The total collection is ~1.4 TB.
Each tiff is preprocessed into a Cloud-Optimised Geotiff using GDAL (see GDAL help). We chose lossless DEFLATE compression as we weren’t sure what the potential future use of the files would be (there was a whole processing pipeline involved to assemble the stitched mosaics from thousands of tiny tiles with custom CRS’s). But as Ian’s second link indicates, JPEG compression may be well suited to your imagery. We used the default COG pyramids, but there are many processing options available.
Converting your geotiffs to COG format will be essential. When you say you have “calculated pyramids”, have you just build pyramids for your tiffs, or are they written as compliant COGs? For fast GeoServer performance with large tiffs, proper COGs are really important. These use internal tiling, embedded overviews, HTTP range requests, small targeted reads rather than full-file reads, header/metadata layout that supports early access, and compression suited to tiled random access.
To publish as an imagemosaic in GeoServer, we accepted all defaults except for the following changes (see Geoserver help here):
- footprint behaviour = cut
(avoids non-transparent areas being rendered outside of the granules)
- set’ ‘maximum number of granules to load’ = 100
(coverage is all of Aus and the individual granules / tiffs themselves cover only a very small spatial area, so there was no need to ever load all granules at once)
- disabled ‘Use deferred loading ImageRead’.
This setting controls how GeoServer reads the individual granules in the mosaic. When enabled, GeoServer streams data and only loads what’s needed during processing. When disabled, GeoServer reads directly and loads the requested data into memory. You may want to experiment with this - originally each of our tifs was sliced up into thousands of smaller tiles, and having this setting enabled was causing “too many open files” errors. If your GeoServer instance doesn’t have much memory available, you may want to enable this instead. 120 granules shouldn’t cause a “too many open files” issues.
This is our single imagemosaic layer [openlayers link] - you won’t see anything here as the granules are tiny (it’s hard to “find” them), but we use this to CQL filter the layer based on a number of custom properties we’ve added to the imagemosaic footprint shapefile.
This is a layer group of the same single imagemosaic layer, with a second vector layer (zoom-based styling) to help locate the small granules [openlayers link]. You can see this is pretty quick.
This imagemosaic WMS is used for annotating seafloor biota in a downstream platform, so it needs to be fairly zippy. We’ve not had any issues with this configuration.
If useful, the entire collection used to generate the single imagemosaic layer is here.
Hope this helps. Happy to share any more of our learnings.
Emma