[Geoserver-devel] Community proposal for back-end GridCoverage distributed caching

Hi all,
I would like to submit the proposal of a new community module to add a distributed caching level to the GeoServer back-end when serving GridCoverages.

This means that raster data composing GridCoverages will be cached in order to avoid repeated read accesses when serving same data. This will reduce time consumption needed to go to the fileSystem/remote disks to get the data bytes or access the DB, parse metadata (CRS, format tags), perform subsampling/oversampling on the read data, assemble different pieces of different images to create the coverage (as for the ImageMosaic case made of multiple granules).

This will be supported by adding a pluggable module (as an extension point) implementing a GridCoverageReaderCallback interface which provides CachingGridCoverageReader instances (implementing GridCoverage2DReader) to the ResourcePool.

Whenever a read operation is performed, it leverages on the cache to setup the requested GridCoverage. Raster data (RenderedImage) composing the GridCoverage are managed through GeoWebCache entities (TileLayer, GridSet, GridSetBroker, BlobStore and derived ones) so we can leverage on well-known/well-tested paradigms.

The underlying coverage read operations (which occurs when data is not available in cache) will be made through WCS requests (in order to get raw GridCoverage data satisfying the requested parameters). Therefore, the caching machinery will deal with a WCSLayer object (extending the abstract GWC TileLayer class), which will retrieve ConveyorTile objects used by the reader to be assembled as parts of the requested coverage.

Last step will be integrating a distributed caching mechanism (as Hazelcast) on the underlying BlobStore structures: that is already under development as part of a work Nicola is doing on GeoWebCache 1.

More documentation will be ready in a couple of hours.

Feedbacks are welcome.
Daniele Romagnoli

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

Sounds interesting, and explains what that memory blob store proposal was all about. The GWC aspects you’ve described seem reasonable . Any thoughts on providing a general WCS caching capability to stand alone GWC similar to the WMS with TileFuser capability it has now?

Hi Kevin,
sorry for the delay.

···

On Wed, Oct 29, 2014 at 9:18 PM, Kevin Smith <ksmith@anonymised.com> wrote:

Sounds interesting, and explains what that memory blob store proposal was all about. The GWC aspects you’ve described seem reasonable .

Any thoughts on providing a general WCS caching capability to stand alone GWC similar to the WMS with TileFuser capability it has now?

Not yet. We will discuss it internally.

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.