[Geoserver-devel] Interesting bit from OGC API - Tiles... storing meta-info about tiles?

GeoServer GeoServer Developer
1

Hi all,
while reviewing the OGC API Tiles spec, found these bits about returning HTTP headers,
telling the client that a given portion of the tile cache might be fully empty, or completely solid,
from that point on:

https://github.com/opengeospatial/ogcapi-tiles/blob/master/core/standard/recommendations/tileset/REC_tc-deepfullempty.adoc

If we had to implement this (it’s just a recommendation) then we’d need to store meta-information around
the tile, something the BlobStore interface and TileObject have no room for right now.

FileBlobStore/AWS wise I guess sidecar files could be used… for some formats headers could be an option too,
but don’t think every tile format allows for that option, or might have limitation regarding the type and size
of info that can be stored in the headers.

Has anyone worked on something like this before?

Cheers
Andrea

···

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

2

If that’s all the meta info to store, anything not in the tile file itself would be overkill.
Looks like we only need 2 bits for a full/empty/unknown flag.
On Linux and MacOS it should be possible to use extended file attributes, dunno about Windows.
I’m pretty sure S3 also supports custom object metadata.

Not fond of the idea of side-car files, though could be an opt-in compromise due to overhead (would require a check for existence for all tiles).

2c.

···

Gabriel Roldán

3

On Saturday, 29 May 2021 5:07:44 AM AEST Gabriel Roldan wrote:

If that's all the meta info to store, anything not in the tile file itself
would be overkill.

On Thu, 27 May 2021 at 07:47, Andrea Aime <andrea.aime@anonymised.com>

wrote:
> Hi all,
> while reviewing the OGC API Tiles spec, found these bits about returning
> HTTP headers,
> telling the client that a given portion of the tile cache might be fully
> empty, or completely solid,
> from that point on:
>
>
> https://github.com/opengeospatial/ogcapi-tiles/blob/master/core/standard/r
> ecommendations/tileset/REC_tc-deepfullempty.adoc
>
> If we had to implement this (it's just a recommendation) then we'd need to
> store meta-information around
> the tile, something the BlobStore interface and TileObject have no room
> for right now.

I'd see it as something that could be useful when we're generating from
vectors. So instead of keeping it associated with the Tile, you'd cache it to
the side (somewhere) and use it on both the server side to save work and pass
it down to the client side if everyone does the extension.

Brad

4

If that’s all the meta info to store, anything not in the tile file itself would be overkill.

Yeah, but formats won’t have, in general, a dedicated “user extra” sections in their format.
Like, PNG only allows for a string title for example, GeoTIFF on the other end allows for custom tags freely.
No idea if vector tiles allow for anything not foreseen, I’m afraid not.

Another option could be to add extra bits at the beginning or end of file, which the tile reader would remove.
However, this approach breaks static tile generation (generate once, move to a static file server).

Looks like we only need 2 bits for a full/empty/unknown flag.
On Linux and MacOS it should be possible to use extended file attributes, dunno about Windows.

Good idea. Not sure about Windows either.

I’m pretty sure S3 also supports custom object metadata.

Not fond of the idea of side-car files, though could be an opt-in compromise due to overhead (would require a check for existence for all tiles).

Yes, should probably be used as a last resort only.

Cheers
Andrea

···

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

5

It’s workable on rastes too, if the code can recognize the requested area is all nodata
or outside of the raster footprint (think nodata mask or vector mask). Or think of a mosaic, and the
request is hitting an area that’s not covered by any image.

Cheers
Andrea

···

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.