[Geoserver-users] Adding grib store fails >= v2.7.1

GeoServer GeoServer User
1

Hi all,

I updated to 2.7.1.1 and can no longer add grib stores.

I’m fairly certain that this is a bug, because I downloaded vanilla versions of 2.6.1, 2.7.0, 2.7.1.1, 2.7-SNAPSHOT and 2.8-SNAPSHOT, added in the correct version of the grib plugin for each of them, and added a single grib store. No other configuration or plugins were added.

2.6.1 and 2.7.0 add the store ok, however the versions above that fail with the following message:

“Could not list layers for this store, an error occurred retrieving them: Failed to create reader from file:data/Pacific.grb and hints null”

Through my days of trying to figure it out I’ve had similar results on CentOS, Amazon Linux, and OSX, all running the binary Geoserver builds.

Is there something that changed around then that might be affecting this?

Ful stack trace is attached.

Thanks & regards,

Miles

stack_trace.txt (22.6 KB)

2

On 26 Jul 2015, at 1:22 am, Miles Jordan <miles@anonymised.com> wrote:

I updated to 2.7.1.1 and can no longer add grib stores.

“Could not list layers for this store, an error occurred retrieving them: Failed to create reader from file:data/Pacific.grb and hints null”

I found this open bug: https://osgeo-org.atlassian.net/browse/GEOS-7115 and it seems to be the same thing, so I’l add some info there.

-Miles

3

On 26 Jul 2015, at 1:41 am, Miles Jordan <miles@anonymised.com> wrote:

On 26 Jul 2015, at 1:22 am, Miles Jordan <miles@anonymised.com> wrote:

I updated to 2.7.1.1 and can no longer add grib stores.

“Could not list layers for this store, an error occurred retrieving them: Failed to create reader from file:data/Pacific.grb and hints null”

I found this open bug: https://osgeo-org.atlassian.net/browse/GEOS-7115 and it seems to be the same thing, so I’l add some info there.

Doing some more looking into this.

I cloned the git repo and set up eclipse to run geoserver with the following dependencies on the class path in the run configuration:

  • the grib project + its automatically added dependencies
  • jai*.jar
  • gt-netcdf jar
  • gt-grib jar

I ran from within Eclipse and it worked… and then I realised my mistake (for want of a better word). I had included versions of gt-grib and gt-netcdf from another folder, from a build on July 13, 2015 - but that had made it work. So it looks like this is a geotools problem.

I confirmed by replacing with the latest versions of these jars and figured out that it was only the gt-grib jar that was causing the issue. So the problem is likely to be in there somewhere, or one of its dependencies.

This seems to be the only thing applicable that was merged around that time: https://github.com/geotools/geotools/pull/906 (upgrading NetCDF)

I can’t immediately identify what would be causing it, but does seem to be related to upgrading NetCDF. Maybe someone else with a bit more knowledge of it would be able to easily see what’s happening.

Cheers,

Miles

4

Hi Miles,
I have created a couple of pull requests to get the zip archives fixed in the next nigthly builds for 2.7.x and 2.8.x.
Hopefully, tomorrow’s nightly build should contain the updated versions.
Do you have any chance to test it again tomorrow and let us know?

I have created a new JIRA since GEOS-7115 refers to issues in mosaicking GeoTIFFs.
https://osgeo-org.atlassian.net/browse/GEOS-7126

Hope this helps.
Cheers,
Daniele

···

On Sun, Jul 26, 2015 at 2:09 PM, Miles Jordan <miles@anonymised.com> wrote:

On 26 Jul 2015, at 1:41 am, Miles Jordan <miles@anonymised.com> wrote:

On 26 Jul 2015, at 1:22 am, Miles Jordan <miles@anonymised.com> wrote:

I updated to 2.7.1.1 and can no longer add grib stores.

“Could not list layers for this store, an error occurred retrieving them: Failed to create reader from file:data/Pacific.grb and hints null”

I found this open bug: https://osgeo-org.atlassian.net/browse/GEOS-7115 and it seems to be the same thing, so I’l add some info there.

Doing some more looking into this.

I cloned the git repo and set up eclipse to run geoserver with the following dependencies on the class path in the run configuration:

  • the grib project + its automatically added dependencies
  • jai*.jar
  • gt-netcdf jar
  • gt-grib jar

I ran from within Eclipse and it worked… and then I realised my mistake (for want of a better word). I had included versions of gt-grib and gt-netcdf from another folder, from a build on July 13, 2015 - but that had made it work. So it looks like this is a geotools problem.

I confirmed by replacing with the latest versions of these jars and figured out that it was only the gt-grib jar that was causing the issue. So the problem is likely to be in there somewhere, or one of its dependencies.

This seems to be the only thing applicable that was merged around that time: https://github.com/geotools/geotools/pull/906 (upgrading NetCDF)

I can’t immediately identify what would be causing it, but does seem to be related to upgrading NetCDF. Maybe someone else with a bit more knowledge of it would be able to easily see what’s happening.

Cheers,

Miles

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

5

Hi Daniele,

Getting there :slight_smile:

It works for gribs that have a temporal dimension, but not for single dimension gribs.

I’ll add an example data file (that works on 2.7.0) and the new stack trace to the JIRA issue.

Thanks for trying to sort it out. It’s important that we are above 2.7.1 for the security fix, otherwise I’d just keep on 2.7.0.

Miles

···

On Sun, Jul 26, 2015 at 2:09 PM, Miles Jordan <miles@anonymised.com> wrote:

On 26 Jul 2015, at 1:41 am, Miles Jordan <miles@anonymised.com> wrote:

On 26 Jul 2015, at 1:22 am, Miles Jordan <miles@anonymised.com> wrote:

I updated to 2.7.1.1 and can no longer add grib stores.

“Could not list layers for this store, an error occurred retrieving them: Failed to create reader from file:data/Pacific.grb and hints null”

I found this open bug: https://osgeo-org.atlassian.net/browse/GEOS-7115 and it seems to be the same thing, so I’l add some info there.

Doing some more looking into this.

I cloned the git repo and set up eclipse to run geoserver with the following dependencies on the class path in the run configuration:

  • the grib project + its automatically added dependencies
  • jai*.jar
  • gt-netcdf jar
  • gt-grib jar

I ran from within Eclipse and it worked… and then I realised my mistake (for want of a better word). I had included versions of gt-grib and gt-netcdf from another folder, from a build on July 13, 2015 - but that had made it work. So it looks like this is a geotools problem.

I confirmed by replacing with the latest versions of these jars and figured out that it was only the gt-grib jar that was causing the issue. So the problem is likely to be in there somewhere, or one of its dependencies.

This seems to be the only thing applicable that was merged around that time: https://github.com/geotools/geotools/pull/906 (upgrading NetCDF)

I can’t immediately identify what would be causing it, but does seem to be related to upgrading NetCDF. Maybe someone else with a bit more knowledge of it would be able to easily see what’s happening.

Cheers,

Miles

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

6

Actually this is another NetCDF issue that I am facing now. I’ll do some research on it tomorrow. The issue that you opened I can confirm is working properly.

Thanks!

Miles

···

On Sun, Jul 26, 2015 at 2:09 PM, Miles Jordan <miles@anonymised.com> wrote:

On 26 Jul 2015, at 1:41 am, Miles Jordan <miles@anonymised.com6428…> wrote:

On 26 Jul 2015, at 1:22 am, Miles Jordan <miles@anonymised.com> wrote:

I updated to 2.7.1.1 and can no longer add grib stores.

“Could not list layers for this store, an error occurred retrieving them: Failed to create reader from file:data/Pacific.grb and hints null”

I found this open bug: https://osgeo-org.atlassian.net/browse/GEOS-7115 and it seems to be the same thing, so I’l add some info there.

Doing some more looking into this.

I cloned the git repo and set up eclipse to run geoserver with the following dependencies on the class path in the run configuration:

  • the grib project + its automatically added dependencies
  • jai*.jar
  • gt-netcdf jar
  • gt-grib jar

I ran from within Eclipse and it worked… and then I realised my mistake (for want of a better word). I had included versions of gt-grib and gt-netcdf from another folder, from a build on July 13, 2015 - but that had made it work. So it looks like this is a geotools problem.

I confirmed by replacing with the latest versions of these jars and figured out that it was only the gt-grib jar that was causing the issue. So the problem is likely to be in there somewhere, or one of its dependencies.

This seems to be the only thing applicable that was merged around that time: https://github.com/geotools/geotools/pull/906 (upgrading NetCDF)

I can’t immediately identify what would be causing it, but does seem to be related to upgrading NetCDF. Maybe someone else with a bit more knowledge of it would be able to easily see what’s happening.

Cheers,

Miles

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

7

I was wrong. I think that the netcdf upgrade introduced a regression. I can no longer use gribs that have time-steps in them.

I filed a bug report here https://osgeo-org.atlassian.net/browse/GEOS-7134

It also has an example grib in the bug report. Can confirm that the grib works with Geoserver < 2.7.1 but no longer.

I tried fixing it myself but I think it’s a bit complex, without having looked at the code for a while, to actually do it correctly. Would appreciate some guidance!

Regards,

Miles

···

On Sun, Jul 26, 2015 at 2:09 PM, Miles Jordan <miles@anonymised.com> wrote:

On 26 Jul 2015, at 1:41 am, Miles Jordan <miles@anonymised.com6428…> wrote:

On 26 Jul 2015, at 1:22 am, Miles Jordan <miles@anonymised.com> wrote:

I updated to 2.7.1.1 and can no longer add grib stores.

“Could not list layers for this store, an error occurred retrieving them: Failed to create reader from file:data/Pacific.grb and hints null”

I found this open bug: https://osgeo-org.atlassian.net/browse/GEOS-7115 and it seems to be the same thing, so I’l add some info there.

Doing some more looking into this.

I cloned the git repo and set up eclipse to run geoserver with the following dependencies on the class path in the run configuration:

  • the grib project + its automatically added dependencies
  • jai*.jar
  • gt-netcdf jar
  • gt-grib jar

I ran from within Eclipse and it worked… and then I realised my mistake (for want of a better word). I had included versions of gt-grib and gt-netcdf from another folder, from a build on July 13, 2015 - but that had made it work. So it looks like this is a geotools problem.

I confirmed by replacing with the latest versions of these jars and figured out that it was only the gt-grib jar that was causing the issue. So the problem is likely to be in there somewhere, or one of its dependencies.

This seems to be the only thing applicable that was merged around that time: https://github.com/geotools/geotools/pull/906 (upgrading NetCDF)

I can’t immediately identify what would be causing it, but does seem to be related to upgrading NetCDF. Maybe someone else with a bit more knowledge of it would be able to easily see what’s happening.

Cheers,

Miles

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

8

Hey everyone,

Apologies up front for going on about this as I know its not high on the list of priorities at the moment.

I feel like the grib store should be excluded from release again until it can be fixed though, because it’s hardly usable from what I can tell, unless you have the most basic of gribs.

As I mentioned I’m happy to look into the problem if someone can guide me.

Another option would be to roll back the netcdf library upgrade until it’s been properly resolved.

Any thoughts or preferences?

Regards,

Miles

···

On Sun, Jul 26, 2015 at 2:09 PM, Miles Jordan <miles@anonymised.com> wrote:

On 26 Jul 2015, at 1:41 am, Miles Jordan <miles@anonymised.com6428…> wrote:

On 26 Jul 2015, at 1:22 am, Miles Jordan <miles@anonymised.com> wrote:

I updated to 2.7.1.1 and can no longer add grib stores.

“Could not list layers for this store, an error occurred retrieving them: Failed to create reader from file:data/Pacific.grb and hints null”

I found this open bug: https://osgeo-org.atlassian.net/browse/GEOS-7115 and it seems to be the same thing, so I’l add some info there.

Doing some more looking into this.

I cloned the git repo and set up eclipse to run geoserver with the following dependencies on the class path in the run configuration:

  • the grib project + its automatically added dependencies
  • jai*.jar
  • gt-netcdf jar
  • gt-grib jar

I ran from within Eclipse and it worked… and then I realised my mistake (for want of a better word). I had included versions of gt-grib and gt-netcdf from another folder, from a build on July 13, 2015 - but that had made it work. So it looks like this is a geotools problem.

I confirmed by replacing with the latest versions of these jars and figured out that it was only the gt-grib jar that was causing the issue. So the problem is likely to be in there somewhere, or one of its dependencies.

This seems to be the only thing applicable that was merged around that time: https://github.com/geotools/geotools/pull/906 (upgrading NetCDF)

I can’t immediately identify what would be causing it, but does seem to be related to upgrading NetCDF. Maybe someone else with a bit more knowledge of it would be able to easily see what’s happening.

Cheers,

Miles

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

9

On Thu, Aug 13, 2015 at 4:39 AM, Miles Jordan <miles@anonymised.com>
wrote:

Hey everyone,

Apologies up front for going on about this as I know its not high on the
list of priorities at the moment.

I feel like the grib store should be excluded from release again until it
can be fixed though, because it’s hardly usable from what I can tell,
unless you have the most basic of gribs.

I don't see it in the release?
http://geoserver.org/release/stable/

As far as I can see it's part of the community modules, which provide no
guarantee whatsoever on
their functionality, and are not officially maintained either:
http://ares.boundlessgeo.com/geoserver/2.7.x/community-latest/

As long as those work for _someone_ I don't see a problem keeping them
there, they are just
made available for people to play with.

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------