[Geoserver-users] Failed to create reader from netCDF

Langer_Christian · November 24, 2022, 12:47pm

Dear geoserver community,

we are a team from a scientific institute in Leipzig/Germany and work with geospatial data, especially with netCDFs files.

We would like to create web services from netCDFs files with the geoserver netcdf plugin, unfortunately the plugin cannot read our netCDFs structure.

To get an idea of the structure of the netCDFs you can check this page: https://git.idiv.de/lq39quba/ebv_data_standard

Two datasets that we propose for ‘taking a look’ are the following:

We are using GeoServer 2.20.1 with the netCDF plugin (geoserver-2.20.1-netcdf-plugin.zip) on a Tomcat 9 locally on a Mac (localhost:8080).

We have followed the tutorial: https://geoserver.geo-solutions.it/multidim/multidim/netcdf/netcdf_config.html#netcdf

We are getting always the error: “Could not list layers for this store, an error occurred retrieving them: Failed to create reader from file” (please see screenshot attached). It seems geoserver has problems to handle the 4th dimension. Complete log file here: geoserver.log

We have tried the same with a netCDF structure in 3d and it works like a charm - geoserver could list the layers (see screenshot attached).

Could we get a contact to the developer of the netCDF Plugin? Or can you help us directly?

Any help is much appreciated!

Thanks in advance.

Best regards,

Christian Langer

Christian Langer

Web-GIS Programmer / Web-GIS Programmierer

German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig

Phone +49 341 9733141

Office B.02.11

Email christian.langer@…11648…

Website www.idiv.de/christian-langer

Affiliations

Martin Luther University Halle-Wittenberg

German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig

Puschstraße 4

04103 Leipzig

Germany

iDiv is a research centre of the DFG – Deutsche Forschungsgemeinschaft

iDiv ist eine zentrale Einrichtung der Universität Leipzig im Sinne des § 92 Abs. 1 SächsHSFG und wird zusammen mit der Martin-Luther-Universität Halle-Wittenberg und der Friedrich-Schiller-Universität Jena betrieben sowie in Kooperation mit dem Helmholtz-Zentrum für Umweltforschung GmbH – UFZ. Beteiligte Kooperationspartner sind die folgenden außeruniversitären Forschungseinrichtungen: das Helmholtz-Zentrum für Umweltforschung GmbH - UFZ, das Max-Planck-Institut für Biogeochemie (MPI BGC), das Max-Planck-Institut für chemische Ökologie (MPI CE), das Max-Planck-Institut für evolutionäre Anthropologie (MPI EVA), das Leibniz-Institut Deutsche Sammlung von Mikroorganismen und Zellkulturen (DSMZ), das Leibniz-Institut für Pflanzenbiochemie (IPB), das Leibniz-Institut für Pflanzengenetik und Kulturpflanzenforschung (IPK) und das Leibniz-Institut Senckenberg Museum für Naturkunde Görlitz (SMNG). USt-IdNr. DE 141510383

Andrea_Aime4 · December 3, 2022, 8:18pm

Hi,
I had a quick look and indeed Geoserver is not able to open the files, I see errors like:

Caused by: org.geotools.data.DataSourceException: java.io.IOException: java.lang.IllegalArgumentException: No valid coverages name to be written
at org.geotools.coverage.io.netcdf.NetCDFAccess.(NetCDFAccess.java:130)
at org.geotools.coverage.io.netcdf.NetCDFDriver.connect(NetCDFDriver.java:93)
at org.geotools.coverage.io.impl.DefaultFileDriver.process(DefaultFileDriver.java:231)
at org.geotools.coverage.io.netcdf.NetCDFReader.(NetCDFReader.java:156)
… 164 more
Caused by: java.lang.RuntimeException: java.io.IOException: java.lang.IllegalArgumentException: No valid coverages name to be written
at org.geotools.imageio.netcdf.NetCDFImageReader.setInput(NetCDFImageReader.java:307)
at java.desktop/javax.imageio.ImageReader.setInput(ImageReader.java:380)
at org.geotools.coverage.io.netcdf.NetCDFAccess.(NetCDFAccess.java:119)
… 167 more
Caused by: java.io.IOException: java.lang.IllegalArgumentException: No valid coverages name to be written
at org.geotools.imageio.netcdf.NetCDFImageReader.initIndex(NetCDFImageReader.java:414)
at org.geotools.imageio.netcdf.NetCDFImageReader.init(NetCDFImageReader.java:541)
at org.geotools.imageio.netcdf.NetCDFImageReader.setInput(NetCDFImageReader.java:305)
… 169 more
Caused by: java.lang.IllegalArgumentException: No valid coverages name to be written
at org.geotools.imageio.netcdf.AncillaryFileManager.storeIndexer(AncillaryFileManager.java:390)
at org.geotools.imageio.netcdf.AncillaryFileManager.writeToDisk(AncillaryFileManager.java:382)
at org.geotools.imageio.netcdf.NetCDFImageReader.initIndex(NetCDFImageReader.java:402)
… 171 more

According to the documentation, GeoServer should be able to open files conforming to the COARDS convention…
checking the files with PanoplyJ I see a grouping/netsting construct I haven’t seen before, which does not seem to appear
in COARDS either:

It might be possible to have GeoServer open these files, but I guess that would need some code
modifications in the NetCDF driver.

Cheers
Andrea

···

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

Langer_Christian · December 19, 2022, 2:28pm

Hi Matthew, Hi Andrea,

Thank you for your answers!

Unfortunately we haven’t found a solution yet why geoserver has problems to create webservices from a netCDF data file with a 4d structure (lat, lon, time, entity).

Still we can’t add the netCDF files as “netcdf store” in the geoserver to list the layers.

@Andrea

The nesting/grouping doesn’t seem to be the problem, since geoserver can create webservices given a 3dimensional netCDF structure (lat, lon, time) with the same nesting.

@Matthew

we have modified the structure from the netCDF as you described. Would it be possible to share your 4d netCDF with us so that we can compare?

Description of netCDFs

All:

4D format, with metric group but without scenario group
the two cubes have different orders for the dimensions, to test that

martins_v1.nc → here

has no attributes at all (only CRS has two: ‘GeoTransform’ and ‘spatial_ref’ and the cubes each have a ‘grid_mapping’ attribute that points to CRS)
metric_1/ebv_cube: pure order of dimensions: “entity”, “time”, “lat”, “lon”
metric_2/ebv_cube: pure order of dimensions: “time”, “entity”, “lat”, “lon” (COARDS makes me think so)

martins_v2.nc → here

has the same attributes as the stackexchange example (+ ‘axis’ because of COARDS)
metric_1/ebv_cube: pure sequence of dimensions: “entity”, “time”, “lat”, “lon”
metric_2/ebv_cube: pure order of dimensions: “time”, “entity”, “lat”, “lon” (COARDS makes me think so)

Resources:

https://gis.stackexchange.com/questions/444317/geoserver-java-error-when-adding-layer-from-netcdf-based-store/444686#444686

https://geos-chem.readthedocs.io/en/latest/geos-chem-shared-docs/supplemental-guides/coards-guide.html

Error message:

Could not list layers for this store, an error occurred retrieving them: Failed to create reader from file:data/netCDF/martins_v1.nc and hints Hints: REPOSITORY = org.geoserver.catalog.CatalogRepository@…11690… EXECUTOR_SERVICE = [java.util.concurrent.ThreadPoolExecutor@…11691…Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0] System defaults: COMPARISON_TOLERANCE = 1.0E-8 FILTER_FACTORY = FilterFactoryImpl STYLE_FACTORY = StyleFactoryImpl FEATURE_FACTORY = org.geotools.feature.LenientFeatureFactoryImpl@…11692… GRID_COVERAGE_FACTORY = GridCoverageFactory TILE_ENCODING = null FORCE_LONGITUDE_FIRST_AXIS_ORDER = true FORCE_AXIS_ORDER_HONORING = http LENIENT_DATUM_SHIFT = true

Is there anyone in the community who can give us contact to the netcdf plugin developer?

Any further help from you and the community is much appreciated!

Thanks!

Best

Christian

···

Von: Matthew Levey <mlevey@…11693…>
Organisation: SeaSpatial Consulting
Datum: Montag, 28. November 2022 um 20:12
An: Christian Langer <christian.langer@…11648…>
Betreff: [Extern] Re: [Geoserver-users] Failed to create reader from netCDF

Hi Christian,

I ran into a similar problem using surface current data from Copernicus (marine.copernicus.eu). I was using xarray and found the solution was to copy the variable attributes of a known working NetCDF file. Here is a link to my question and self-answer on GIS StackExchange: https://gis.stackexchange.com/questions/444317/geoserver-java-error-when-adding-layer-from-netcdf-based-store/444686#444686

I hope this helps!

regards,

Matt Levey

On 11/24/2022 4:47 AM, Langer, Christian wrote:

Dear geoserver community,

we are a team from a scientific institute in Leipzig/Germany and work with geospatial data, especially with netCDFs files.

We would like to create web services from netCDFs files with the geoserver netcdf plugin, unfortunately the plugin cannot read our netCDFs structure.

To get an idea of the structure of the netCDFs you can check this page: https://git.idiv.de/lq39quba/ebv_data_standard

Two datasets that we propose for ‘taking a look’ are the following:

https://portal.geobon.org/data/upload/1/public/martins_comcom_id1_20220208_v1.nc

https://portal.geobon.org/data/upload/5/public/baisero_spepop_id5_20220215_v2.nc

We are using GeoServer 2.20.1 with the netCDF plugin (geoserver-2.20.1-netcdf-plugin.zip) on a Tomcat 9 locally on a Mac (localhost:8080).

We have followed the tutorial: https://geoserver.geo-solutions.it/multidim/multidim/netcdf/netcdf_config.html#netcdf

We are getting always the error: “Could not list layers for this store, an error occurred retrieving them: Failed to create reader from file” (please see screenshot attached). It seems geoserver has problems to handle the 4th dimension. Complete log file here: geoserver.log

We have tried the same with a netCDF structure in 3d and it works like a charm - geoserver could list the layers (see screenshot attached).

Could we get a contact to the developer of the netCDF Plugin? Or can you help us directly?

Any help is much appreciated!

Thanks in advance.

Best regards,

Christian Langer

Christian Langer

Web-GIS Programmer / Web-GIS Programmierer

German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig

–