[Geoserver-users] Rasters and time...again

Hi all,

I’ve been browsing the archives looking for pointers on getting some time-series raster data published. And as usual, I’m left with noob questions :-).

Background: I’m looking to publish results of various weather-derived models. The outputs are at 15-kilometer resolution, and cover about 800 by 1000 kilometers. The models run every day and we have data going back 15 years. So by the time I retire we’re talking 10,000 rasters of that size. (Or more; the National Weather Service is contemplating 4K resolution!)

I had assumed that I’d be using PostGIS rasters for this. It seemed improbable that GeoServer would nimbly handle 10,000 files for each model (we have about 8 of them). First question: Is that correct, or will it be happy managing all those files? (Obviously I’d have to use the REST API to manage them!) Since the ImageMosaicJDBC plugin doesn’t handle timesteps, that might not work at all for me.

Second question: the image mosaic time series tutorial publishes a fixed set of TIFs. Just to be clear: as TIFs with new timestamps are added to the store, do they automatically become available?

Thirdly, said tutorial ends with “Create and publish a Layer from mosaic indexes”. I’m not getting the intent there – a combination of my thickness and the tutorial written by someone for whom English is not their first language. Don’t get me wrong, it’s overall very clear and I’m grateful to the author! (I’m sure my competence in their native language is laughable at best.) But I’m baffled by what this particular step does – what are the “footprints of the images”, and why would the user want to get them? This has to do only with spatial indexing, and nothing to do with the time dimension, correct?

Finally, there’s one possible optimization: Although we need to keep the data going back for basically ever, and need to be able to do various calculations on it and map the results by hand, if push comes to shove, we only have to present the most recent visualizations automatically on the Web. So for example, perhaps I can use the IMJDBC plugin if I add a timestamp column to the raster table, create a “most recent” table, and point IMJDBC at a view that does something like “…where raster_table.timestamp = most_recent.timestamp”. Does anybody know if that will work? The alternative would be to just use a file-based raster for the current day, and keep overwriting it.

Thanks, yet again, for your help. And just so you know, GeoServer enables a couple of applications that have pretty high political visibility and are making me look quite brilliant to my stakeholders* – so thanks to the GeoServer team!

rw


*Only you folks know the truth! Shhhhh! >;-}

Rick

···

Background: I’m looking to publish results of various weather-derived models. The outputs are at 15-kilometer resolution, and cover about 800 by 1000 kilometers. The models run every day and we have data going back 15 years. So by the time I retire we’re talking 10,000 rasters of that size. (Or more; the National Weather Service is contemplating 4K resolution!)

The ImageMosaic can handle this kind collection quite easily, I’m currently working on a mosaic with 300K rasters.

I had assumed that I’d be using PostGIS rasters for this. It seemed improbable that GeoServer would nimbly handle 10,000 files for each model (we have about 8 of them). First question: Is that correct, or will it be happy managing all those files? (Obviously I’d have to use the REST API to manage them!) Since the ImageMosaicJDBC plugin doesn’t handle timesteps, that might not work at all for me.

I suggest to use the ImageMosaic plugin.

Second question: the image mosaic time series tutorial publishes a fixed set of TIFs. Just to be clear: as TIFs with new timestamps are added to the store, do they automatically become available?

Sure, anyhow sometime you may have to refresh the store depending on the BBox of the granule you have added (This happens since GeoServer caches readers).

Thirdly, said tutorial ends with “Create and publish a Layer from mosaic indexes”. I’m not getting the intent there – a combination of my thickness and the tutorial written by someone for whom English is not their first language. Don’t get me wrong, it’s overall very clear and I’m grateful to the author! (I’m sure my competence in their native language is laughable at best.) But I’m baffled by what this particular step does – what

are the “footprints of the images”, and why would the user want to get them? This has to do only with spatial indexing, and nothing to do with the time dimension, correct?

Not sure about this anyhow this may be related to the bbox of the images added to the mosaic. You may want to publish a new layer from the obtained table (created by the imagemosaic plugin).
This can be achieved creating a new DataStore pointing to that table.
This layer can then be used to show the content of the mosaic in terms of spatial (and temporal) dimensions, users can then perform light queries (also using an UI like OpenLayers) on this layer to establish which is the area of interest (useful to build a query and actually use it on the mosaic).

Finally, there’s one possible optimization: Although we need to keep the data going back for basically ever, and need to be able to do various calculations on it and map the results by hand, if push comes to shove, we only have to present the most recent

visualizations automatically on the Web. So for example, perhaps I can use the IMJDBC plugin if I add a timestamp column to the raster table, create a “most recent” table, and point IMJDBC at a view that does something like “…where raster_table.timestamp = most_recent.timestamp”. Does anybody know if that will work? The alternative would be to just use a file-based raster for the current day, and keep overwriting it.

Here you mean the last added image (granule)?
with the imagemosaic plugin, when using dimensions, if you do not specify anything into the request the default values are used. In the case of the time dimension it is the latest (more recent).

Thanks, yet again, for your help. And just so you know, GeoServer enables a couple of applications that have pretty high political visibility and are making me look quite brilliant to my stakeholders* – so thanks to the GeoServer team!

You are welcome.

Hope this helps.

Cheers,
Carlo

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Dott. Carlo Cancellieri
@cancellieric
Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
mobile: +39 3371094494
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it