[Geoserver-devel] Mosaics with postgis raster and time

Hi folks,

I have a need to publish a mosaic composed of tiles from a postgis raster table in a time series. From what I can tell for the two mosaic options the situation is:

  • the core imagemosaic reader can’t read tiles from a postgis raster table
  • the imagemosaic-jdbc plugin can’t do time

If either of those assumptions are wrong please let me know, I’m basing that on what i’ve found in the docs and mailing lists, and a quick code review but could have easily missed something.

Assuming those assumptions are both correct I am thinking going the route of adding support for a time dimension to the imagemosaic-jdbc plugin is probably the path of least resistance? Before I start down that path I thought I would reach out to the experts. Any thoughts much appreciated.

Thanks!

-Justin

Hi Justin,
I believe your assessment is correct, also in terms of effort, it should be easier to add time support to the imagemosaic-jdbc module.

I’m however a bit worried about the module, the history shows very little changes and most of them are
side effects of refactors happening elsewhere, which makes me wonder how much “life” still remains in the module:

https://github.com/geotools/geotools/commits/master/modules/plugin/imagemosaic-jdbc/src/main

Cheers
Andrea

···

Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Thanks for the input Andrea. Yeah I’m a bit weary of putting time into the image mosaic jdbc plugin if it’s falling toward unsupported. But I don’t think the project scope will allow for going the route of trying to update the core plugin so it’s a bit of a catch 22. I’m going to talk to the project stakeholders and see what they want to do. I’ll let the list know what comes out of that.

Thanks again!

  • Justin
···

On Nov 12, 2018 2:44 AM, “Andrea Aime” <andrea.aime@anonymised.com> wrote:

Hi Justin,
I believe your assessment is correct, also in terms of effort, it should be easier to add time support to the imagemosaic-jdbc module.

I’m however a bit worried about the module, the history shows very little changes and most of them are
side effects of refactors happening elsewhere, which makes me wonder how much “life” still remains in the module:

https://github.com/geotools/geotools/commits/master/modules/plugin/imagemosaic-jdbc/src/main

Cheers
Andrea

On Tue, Nov 6, 2018 at 9:04 PM Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I have a need to publish a mosaic composed of tiles from a postgis raster table in a time series. From what I can tell for the two mosaic options the situation is:

  • the core imagemosaic reader can’t read tiles from a postgis raster table
  • the imagemosaic-jdbc plugin can’t do time

If either of those assumptions are wrong please let me know, I’m basing that on what i’ve found in the docs and mailing lists, and a quick code review but could have easily missed something.

Assuming those assumptions are both correct I am thinking going the route of adding support for a time dimension to the imagemosaic-jdbc plugin is probably the path of least resistance? Before I start down that path I thought I would reach out to the experts. Any thoughts much appreciated.

Thanks!

-Justin


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Hi Andreas and Justin

Nice to talk with you again. At the time I developed the ImageMosaic-Jdbc module I was focused on Geotools and I did not know about the relation to Geoserver. Maybe my fault. In the meantime it is a problem of time, no chance for me to invest further time. I know that this module is not a “smooth” solution.

Sorry, Christian

···

On Nov 12, 2018 2:44 AM, “Andrea Aime” <andrea.aime@anonymised.com…> wrote:

Hi Justin,
I believe your assessment is correct, also in terms of effort, it should be easier to add time support to the imagemosaic-jdbc module.

I’m however a bit worried about the module, the history shows very little changes and most of them are
side effects of refactors happening elsewhere, which makes me wonder how much “life” still remains in the module:

https://github.com/geotools/geotools/commits/master/modules/plugin/imagemosaic-jdbc/src/main

Cheers
Andrea

On Tue, Nov 6, 2018 at 9:04 PM Justin Deoliveira <jdeolive@anonymised.com403…> wrote:

Hi folks,

I have a need to publish a mosaic composed of tiles from a postgis raster table in a time series. From what I can tell for the two mosaic options the situation is:

  • the core imagemosaic reader can’t read tiles from a postgis raster table
  • the imagemosaic-jdbc plugin can’t do time

If either of those assumptions are wrong please let me know, I’m basing that on what i’ve found in the docs and mailing lists, and a quick code review but could have easily missed something.

Assuming those assumptions are both correct I am thinking going the route of adding support for a time dimension to the imagemosaic-jdbc plugin is probably the path of least resistance? Before I start down that path I thought I would reach out to the experts. Any thoughts much appreciated.

Thanks!

-Justin


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

DI Christian Mueller MSc (GIS), MSc (IT-Security)
OSS Open Source Solutions GmbH

Thanks Christian If anyone understands the issue with lack of time to put toward open source stuff it’s me :slight_smile: I hope to get back to it someday… maybe when my kids are out of the house :wink:

One thing you could help me with is a quick sanity check on the approach. I was basically just planning to add a “timeAttribute” element to the mapping file, and when present have the coverage reader declare the appropriate metadata in support of the time domain. Let me know if you think that is the wrong way to go.

Thanks!

···

On Nov 12, 2018 2:44 AM, “Andrea Aime” <andrea.aime@anonymised.com> wrote:

Hi Justin,
I believe your assessment is correct, also in terms of effort, it should be easier to add time support to the imagemosaic-jdbc module.

I’m however a bit worried about the module, the history shows very little changes and most of them are
side effects of refactors happening elsewhere, which makes me wonder how much “life” still remains in the module:

https://github.com/geotools/geotools/commits/master/modules/plugin/imagemosaic-jdbc/src/main

Cheers
Andrea

On Tue, Nov 6, 2018 at 9:04 PM Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I have a need to publish a mosaic composed of tiles from a postgis raster table in a time series. From what I can tell for the two mosaic options the situation is:

  • the core imagemosaic reader can’t read tiles from a postgis raster table
  • the imagemosaic-jdbc plugin can’t do time

If either of those assumptions are wrong please let me know, I’m basing that on what i’ve found in the docs and mailing lists, and a quick code review but could have easily missed something.

Assuming those assumptions are both correct I am thinking going the route of adding support for a time dimension to the imagemosaic-jdbc plugin is probably the path of least resistance? Before I start down that path I thought I would reach out to the experts. Any thoughts much appreciated.

Thanks!

-Justin


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Sounds reasonable to me. The situation now is a bit fluid, there are some methods accessing dimension domains
that use the StructuredCoverageGridReader interface to allow for certain database optimizations, but the metadata
entries are still supported and in some cases they are the only ones still used.

Some examples of both approaches in this class:
https://github.com/geoserver/geoserver/blob/2e681e2a74f0754e294bbb481ecf7ad33552b3e6/src/main/src/main/java/org/geoserver/catalog/util/ReaderDimensionsAccessor.java

Cheers
Andrea

···

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Ok great. Thanks for the info Andrea!

On Wed, Nov 14, 2018, 11:57 AM Andrea Aime <andrea.aime@anonymised.com wrote:

On Tue, Nov 13, 2018 at 5:30 PM Justin Deoliveira <jdeolive@anonymised.com> wrote:

One thing you could help me with is a quick sanity check on the approach. I was basically just planning to add a “timeAttribute” element to the mapping file, and when present have the coverage reader declare the appropriate metadata in support of the time domain. Let me know if you think that is the wrong way to go.

Sounds reasonable to me. The situation now is a bit fluid, there are some methods accessing dimension domains
that use the StructuredCoverageGridReader interface to allow for certain database optimizations, but the metadata
entries are still supported and in some cases they are the only ones still used.

Some examples of both approaches in this class:
https://github.com/geoserver/geoserver/blob/2e681e2a74f0754e294bbb481ecf7ad33552b3e6/src/main/src/main/java/org/geoserver/catalog/util/ReaderDimensionsAccessor.java

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Hi Justin and Adrea

Sorry for the late

···

DI Christian Mueller MSc (GIS), MSc (IT-Security)
OSS Open Source Solutions GmbH

Thanks for the info Christian, I assume you replied directly to me by accident so re-including the list.

My initial thought was that there would be just one “time column” composed of discrete timestamps, or at least that is the use case I currently have. I am also kind of going off of what the core mosaic reader does and unless I am mistaken (which is very possible to someone please let me know :slight_smile: ) there is just a single time attribute for each entry of the mosaic and you can specify the time resolution to essentially create an interval.

That said a start/end date also makes sense to me. I haven’t started digging into the code yet but I will keep this in mind in the approach and perhaps add support for both modes if people think that makes sense?

Thanks!

On Sun, Nov 18, 2018 at 10:09 AM Christian Mueller <christian.mueller@anonymised.com> wrote:

Hi Justin and Andrea

Sorry for the late reply. IMHO it would make sense to add the business time support as described in SQL 2011. At the end of the day you need 2 attrilbutes of type DATE, one for the start and one for the end. (The open end may be something like 2999-12-31). The important issue is that the start date is included in the time period and the end date is NOT included.

I know about support in DB2 and Oracle, Postgres provides an intervall data type, and I think Andrea mentioned MS Sqlserver some time ago.

Just my 2 cents.

On Sun, Nov 18, 2018 at 6:04 PM Christian Mueller <christian.mueller@anonymised.com> wrote:

Hi Justin and Adrea

Sorry for the late

On Wed, Nov 14, 2018 at 9:14 PM Justin Deoliveira <jdeolive@anonymised.com> wrote:

Ok great. Thanks for the info Andrea!

On Wed, Nov 14, 2018, 11:57 AM Andrea Aime <andrea.aime@anonymised.com wrote:

On Tue, Nov 13, 2018 at 5:30 PM Justin Deoliveira <jdeolive@anonymised.com> wrote:

One thing you could help me with is a quick sanity check on the approach. I was basically just planning to add a “timeAttribute” element to the mapping file, and when present have the coverage reader declare the appropriate metadata in support of the time domain. Let me know if you think that is the wrong way to go.

Sounds reasonable to me. The situation now is a bit fluid, there are some methods accessing dimension domains
that use the StructuredCoverageGridReader interface to allow for certain database optimizations, but the metadata
entries are still supported and in some cases they are the only ones still used.

Some examples of both approaches in this class:
https://github.com/geoserver/geoserver/blob/2e681e2a74f0754e294bbb481ecf7ad33552b3e6/src/main/src/main/java/org/geoserver/catalog/util/ReaderDimensionsAccessor.java

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

DI Christian Mueller MSc (GIS), MSc (IT-Security)
OSS Open Source Solutions GmbH

DI Christian Mueller MSc (GIS), MSc (IT-Security)
OSS Open Source Solutions GmbH

The image mosaic works either off one timestamp column, for instant time support, or two time columns, begin and end, for time range support.

Cheers
Andrea

···

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Ok cool, sorry I didn’t see that in the docs. Thanks Andrea, that’s very helpful.

On Mon, Nov 19, 2018 at 12:13 PM Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Nov 19, 2018 at 8:09 PM Justin Deoliveira <jdeolive@anonymised.com> wrote:

Thanks for the info Christian, I assume you replied directly to me by accident so re-including the list.

My initial thought was that there would be just one “time column” composed of discrete timestamps, or at least that is the use case I currently have. I am also kind of going off of what the core mosaic reader does and unless I am mistaken (which is very possible to someone please let me know :slight_smile: ) there is just a single time attribute for each entry of the mosaic and you can specify the time resolution to essentially create an interval.

That said a start/end date also makes sense to me. I haven’t started digging into the code yet but I will keep this in mind in the approach and perhaps add support for both modes if people think that makes sense?

The image mosaic works either off one timestamp column, for instant time support, or two time columns, begin and end, for time range support.

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Hey folks, just thought I would update on this. So after looking into the imagemosaic jdbc plugin we decided to go with a different approach. Once I started looking into what the effort of updating the imagemosaic jdbc stuff it seemed like much less effort to just implement reader dedicated to postgis rasters. Also much simpler in terms of configuration since it won’t require a bunch of manual mapping, etc… so I’ve embarked on that approach.

The project will be happy to donate all of that code to GeoTools if people find it useful.

Thanks again to Andrea and Christian for all the feedback.

On Mon, Nov 19, 2018 at 12:21 PM Justin Deoliveira <jdeolive@anonymised.com> wrote:

Ok cool, sorry I didn’t see that in the docs. Thanks Andrea, that’s very helpful.

On Mon, Nov 19, 2018 at 12:13 PM Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Nov 19, 2018 at 8:09 PM Justin Deoliveira <jdeolive@anonymised.com> wrote:

Thanks for the info Christian, I assume you replied directly to me by accident so re-including the list.

My initial thought was that there would be just one “time column” composed of discrete timestamps, or at least that is the use case I currently have. I am also kind of going off of what the core mosaic reader does and unless I am mistaken (which is very possible to someone please let me know :slight_smile: ) there is just a single time attribute for each entry of the mosaic and you can specify the time resolution to essentially create an interval.

That said a start/end date also makes sense to me. I haven’t started digging into the code yet but I will keep this in mind in the approach and perhaps add support for both modes if people think that makes sense?

The image mosaic works either off one timestamp column, for instant time support, or two time columns, begin and end, for time range support.

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Hi Justin,
makes sense to me, thanks for keeping up up to date. The code seems interesting, what about putting it in a community module?

Wondering how you’ll manage the connections and their eventual pooling, Gabriel pulled some trick with the SDE to make it share
with the store, could be interesting to have the same, e.g., the ability to pass a pool as parts of the reader hints maybe.

Cheers
Andrea

···

Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Thanks Andrea.

So yeah, re managing the connections the current implementation I have allows you to specify the the connection params directly, ie the reader would manage it’s own connection pool. But also allow for grabbing the connection from JNDI. Basically what the JDBC datastores do. For the project I am going to deploy this in the JNDI option will be what we use to optimize the connection usage.

Passing in a connection pool via a hint would also work nicely, basically the same way it works when an ExecutorService is passed in. I am not sure what the GeoServer side of that looks like though…

It would also be nice to utilize the apis in coverage for serving up multiple coverages from a single reader. I went down that route but was a little confused as to how well this is supported. AbstractGridCoverage2DReader seems to be written to maintain state about multiple coverages which is good. But on the GeoServer side I am unsure if the relationship between coverage store and coverage has to be 1:1… has this changed at all recently? Or maybe it was always possible… not sure. Anyways, if it was possible to use a single reader instance for multiple coverages from the same coverage store storing a connection pool in that reader would also be a good way to share the connection pool.

Interested to hear your thoughts.

-Justin

On Thu, Nov 29, 2018 at 7:51 AM Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi Justin,
makes sense to me, thanks for keeping up up to date. The code seems interesting, what about putting it in a community module?

Wondering how you’ll manage the connections and their eventual pooling, Gabriel pulled some trick with the SDE to make it share
with the store, could be interesting to have the same, e.g., the ability to pass a pool as parts of the reader hints maybe.

Cheers
Andrea

On Tue, Nov 27, 2018 at 11:44 PM Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hey folks, just thought I would update on this. So after looking into the imagemosaic jdbc plugin we decided to go with a different approach. Once I started looking into what the effort of updating the imagemosaic jdbc stuff it seemed like much less effort to just implement reader dedicated to postgis rasters. Also much simpler in terms of configuration since it won’t require a bunch of manual mapping, etc… so I’ve embarked on that approach.

The project will be happy to donate all of that code to GeoTools if people find it useful.

Thanks again to Andrea and Christian for all the feedback.

On Mon, Nov 19, 2018 at 12:21 PM Justin Deoliveira <jdeolive@anonymised.com> wrote:

Ok cool, sorry I didn’t see that in the docs. Thanks Andrea, that’s very helpful.

On Mon, Nov 19, 2018 at 12:13 PM Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Nov 19, 2018 at 8:09 PM Justin Deoliveira <jdeolive@anonymised.com> wrote:

Thanks for the info Christian, I assume you replied directly to me by accident so re-including the list.

My initial thought was that there would be just one “time column” composed of discrete timestamps, or at least that is the use case I currently have. I am also kind of going off of what the core mosaic reader does and unless I am mistaken (which is very possible to someone please let me know :slight_smile: ) there is just a single time attribute for each entry of the mosaic and you can specify the time resolution to essentially create an interval.

That said a start/end date also makes sense to me. I haven’t started digging into the code yet but I will keep this in mind in the approach and perhaps add support for both modes if people think that makes sense?

The image mosaic works either off one timestamp column, for instant time support, or two time columns, begin and end, for time range support.

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Thanks Andrea.

So yeah, re managing the connections the current implementation I have allows you to specify the the connection params directly, ie the reader would manage it’s own connection pool. But also allow for grabbing the connection from JNDI. Basically what the JDBC datastores do. For the project I am going to deploy this in the JNDI option will be what we use to optimize the connection usage.

Passing in a connection pool via a hint would also work nicely, basically the same way it works when an ExecutorService is passed in. I am not sure what the GeoServer side of that looks like though…

Yeah, that would need to be “invented”. Like some setting to pick the connection pool from some other existing store for example.
To be honest, I have been itching to add to GeoServer connection pool as first class catalog objects that can then be passed around to various stores, but never got around to do it.

It would also be nice to utilize the apis in coverage for serving up multiple coverages from a single reader. I went down that route but was a little confused as to how well this is supported. AbstractGridCoverage2DReader seems to be written to maintain state about multiple coverages which is good. But on the GeoServer side I am unsure if the relationship between coverage store and coverage has to be 1:1… has this changed at all recently? Or maybe it was always possible… not sure. Anyways, if it was possible to use a single reader instance for multiple coverages from the same coverage store storing a connection pool in that reader would also be a good way to share the connection pool.

We do manage multilple rasters per reader with mosaic and netcdf without particular problems. If memory serves me right, ResourcePool wraps the readers with a renaming class that gives the rest of the code the illusion there is a single raster in the reader.

Cheers
Andrea

···

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Ahh ok, cool I’ll look into that a bit more closely. I’ve got the code to the point where basic mosaicing is working and it can handle all of the time series stuff and now am going to circle back and clean a few things up.

I’ll definitely put this into a community module. I’ve already had someone from the community reach out expressing interest in the code. I’ll start a separate through on geotools-devel for that soon.

Thanks again for the all of the feedback and help!

On Thu, Nov 29, 2018 at 9:32 AM Andrea Aime <andrea.aime@anonymised.com> wrote:

On Thu, Nov 29, 2018 at 5:27 PM Justin Deoliveira <jdeolive@anonymised.com…403…> wrote:

Thanks Andrea.

So yeah, re managing the connections the current implementation I have allows you to specify the the connection params directly, ie the reader would manage it’s own connection pool. But also allow for grabbing the connection from JNDI. Basically what the JDBC datastores do. For the project I am going to deploy this in the JNDI option will be what we use to optimize the connection usage.

Passing in a connection pool via a hint would also work nicely, basically the same way it works when an ExecutorService is passed in. I am not sure what the GeoServer side of that looks like though…

Yeah, that would need to be “invented”. Like some setting to pick the connection pool from some other existing store for example.
To be honest, I have been itching to add to GeoServer connection pool as first class catalog objects that can then be passed around to various stores, but never got around to do it.

It would also be nice to utilize the apis in coverage for serving up multiple coverages from a single reader. I went down that route but was a little confused as to how well this is supported. AbstractGridCoverage2DReader seems to be written to maintain state about multiple coverages which is good. But on the GeoServer side I am unsure if the relationship between coverage store and coverage has to be 1:1… has this changed at all recently? Or maybe it was always possible… not sure. Anyways, if it was possible to use a single reader instance for multiple coverages from the same coverage store storing a connection pool in that reader would also be a good way to share the connection pool.

We do manage multilple rasters per reader with mosaic and netcdf without particular problems. If memory serves me right, ResourcePool wraps the readers with a renaming class that gives the rest of the code the illusion there is a single raster in the reader.

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.