[Geoserver-users] NetCDF data with 2D coordinate variables

GeoServer GeoServer User
1

Dear fellows,

I have problems with storing NetCDF files with 2D coordinates. NetCDF files with 1D variables can be stored without problem.

Would you please introduce a sample 2D NetCDF data file that can be stored by GeoServer?

Best,
Amirhossein

PS: Bellow are the errors after trying to store the 2D NetCDF file:
Failed to create reader from file:PS_ts.nc and hints Hints: FILTER_FACTORY = FilterFactoryImpl GRID_COVERAGE_FACTORY = GridCoverageFactory TILE_ENCODING = null EXECUTOR_SERVICE = java.util.concurrent.ThreadPoolExecutor@anonymised.com[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0] FORCE_LONGITUDE_FIRST_AXIS_ORDER = true LENIENT_DATUM_SHIFT = true COMPARISON_TOLERANCE = 1.0E-8 REPOSITORY = org.geoserver.catalog.CatalogRepository@anonymised.com FORCE_AXIS_ORDER_HONORING = http STYLE_FACTORY = StyleFactoryImpl FEATURE_FACTORY = org.geotools.feature.LenientFeatureFactoryImpl@anonymised.com

2

Hi,
I believe the NetCDF reader can handle only scalar variables… they can have many dimensions (lon, lat, time, elevation, and custom ones),
but if your variable contains, say, a vector, magnitude and direction for each point, then no, the reader does not support it.

Best regards
Andrea

···

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

3

Well actually in the documentation it has been mentioned that GeoServer can handle NetCDF data with 2D coordinates (latitudes/longitudes).
I just want a sample NetCDF data with 2D coordinates to test it.

Best regards,

Amir

On Thu, May 4, 2023 at 12:14 PM Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
I believe the NetCDF reader can handle only scalar variables… they can have many dimensions (lon, lat, time, elevation, and custom ones),
but if your variable contains, say, a vector, magnitude and direction for each point, then no, the reader does not support it.

Best regards
Andrea

On Tue, May 2, 2023 at 7:09 PM Amirhossein Nikfal <ah.nikfal@anonymised.com> wrote:

Dear fellows,

I have problems with storing NetCDF files with 2D coordinates. NetCDF files with 1D variables can be stored without problem.

Would you please introduce a sample 2D NetCDF data file that can be stored by GeoServer?

Best,
Amirhossein

PS: Bellow are the errors after trying to store the 2D NetCDF file:
Failed to create reader from file:PS_ts.nc and hints Hints: FILTER_FACTORY = FilterFactoryImpl GRID_COVERAGE_FACTORY = GridCoverageFactory TILE_ENCODING = null EXECUTOR_SERVICE = java.util.concurrent.ThreadPoolExecutor@anonymised.com[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0] FORCE_LONGITUDE_FIRST_AXIS_ORDER = true LENIENT_DATUM_SHIFT = true COMPARISON_TOLERANCE = 1.0E-8 REPOSITORY = org.geoserver.catalog.CatalogRepository@anonymised.com FORCE_AXIS_ORDER_HONORING = http STYLE_FACTORY = StyleFactoryImpl FEATURE_FACTORY = org.geotools.feature.LenientFeatureFactoryImpl@anonymised.com

Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

–

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

4

Well actually in the documentation it has been mentioned that GeoServer can handle NetCDF data with 2D coordinates (latitudes/longitudes).

A scalar variable with lat and lon? Sure, it’s supported.
When you said a 2D variable, I understood you wanted a vector-type variable (not sure if NetCDF even supports it, TBH).

I just want a sample NetCDF data with 2D coordinates to test it.

This tiny NetCDF used for the NetCDF module tests should do the trick:
https://github.com/geoserver/geoserver/raw/main/src/extension/netcdf/src/test/resources/org/geoserver/netcdf/test-data/sst.nc

Regards
Andrea

···

GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

5

Thank you very much for the data. However, the file (sst.nc) you sent had 1D coordinates. These kinds of coordinates are okay in GeoServer:

lat_1D: float32 lat(y)
lon_1D: float32 lon(x)

The latitudes and longitudes in a 2D NetCDF data have 2D shapes, and they depend on both the x and y axis. They are problematic in GeoServer, although in the documentations it is mentioned that they can be handled):

lat_2D: float32 lat(x,y)
lon_2D: float32 lon(x,y)

On Thu, May 4, 2023 at 5:09 PM Andrea Aime <andrea.aime@anonymised.com> wrote:

On Thu, May 4, 2023 at 3:09 PM Amirhossein Nikfal <ah.nikfal@anonymised.com> wrote:

Well actually in the documentation it has been mentioned that GeoServer can handle NetCDF data with 2D coordinates (latitudes/longitudes).

A scalar variable with lat and lon? Sure, it’s supported.
When you said a 2D variable, I understood you wanted a vector-type variable (not sure if NetCDF even supports it, TBH).

I just want a sample NetCDF data with 2D coordinates to test it.

This tiny NetCDF used for the NetCDF module tests should do the trick:
https://github.com/geoserver/geoserver/raw/main/src/extension/netcdf/src/test/resources/org/geoserver/netcdf/test-data/sst.nc

Regards
Andrea

==

GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

6

Ah, it’s an irregular grid then, each entry of the array is mapped to its own independent point? No, as far as I know GeoServer has no support for it, regardless of the format (nothing specific to NetCDF). Where did you read those are supported?

I has trouble understanding what you meant so far, it’s likely there is a similar misunderstanding in the docs.

Andrea

Il ven 5 mag 2023, 13:57 Amirhossein Nikfal <ah.nikfal@anonymised.com…> ha scritto:

Thank you very much for the data. However, the file (sst.nc) you sent had 1D coordinates. These kinds of coordinates are okay in GeoServer:

lat_1D: float32 lat(y)
lon_1D: float32 lon(x)

The latitudes and longitudes in a 2D NetCDF data have 2D shapes, and they depend on both the x and y axis. They are problematic in GeoServer, although in the documentations it is mentioned that they can be handled):

lat_2D: float32 lat(x,y)
lon_2D: float32 lon(x,y)

On Thu, May 4, 2023 at 5:09 PM Andrea Aime <andrea.aime@anonymised.com> wrote:

On Thu, May 4, 2023 at 3:09 PM Amirhossein Nikfal <ah.nikfal@anonymised.com.84…> wrote:

Well actually in the documentation it has been mentioned that GeoServer can handle NetCDF data with 2D coordinates (latitudes/longitudes).

A scalar variable with lat and lon? Sure, it’s supported.
When you said a 2D variable, I understood you wanted a vector-type variable (not sure if NetCDF even supports it, TBH).

I just want a sample NetCDF data with 2D coordinates to test it.

This tiny NetCDF used for the NetCDF module tests should do the trick:
https://github.com/geoserver/geoserver/raw/main/src/extension/netcdf/src/test/resources/org/geoserver/netcdf/test-data/sst.nc

Regards
Andrea

==

GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

7

Quoting the documentation:

There is limited support for coordinate variables with two dimensions (see Two-Dimensional Coordinate Variables), as part of the result of an aggregation (such as time,runtime - in the case of a runtime aggregation). Two dimensional non-independent latitude-longitude coordinate variables aren’t currently supported.

See also: https://docs.geoserver.org/stable/en/user/extensions/netcdf/netcdf.html#two-dimensional-coordinate-variables

While the screenshot shows a lat/lon case, the text only refers to time/runtime, and indicates
it ends up flattening the dimension space into a single list. I doubt there is any support for scattered grids (it would be a surprise to me).

Cheers
Andrea

···

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

8

I think it is clearly mentioned in the documentation that NetCDF with 2D lat/lon are supported by GeoServer:
https://docs.geoserver.org/stable/en/user/extensions/netcdf/netcdf.html#two-dimensional-coordinate-variables

I am looking for a sample NetCDF datafile with 2D coordinates (mentioned in documentation) to play with in GeoServer. My own datafiles (with 2D lat/lon) cannot be read by GeoServer.

Best,
Amir

On Fri, May 5, 2023 at 3:06 PM Andrea Aime <andrea.aime@anonymised.com…10949…> wrote:

Ah, it’s an irregular grid then, each entry of the array is mapped to its own independent point? No, as far as I know GeoServer has no support for it, regardless of the format (nothing specific to NetCDF). Where did you read those are supported?

I has trouble understanding what you meant so far, it’s likely there is a similar misunderstanding in the docs.

Andrea

Il ven 5 mag 2023, 13:57 Amirhossein Nikfal <ah.nikfal@anonymised.com> ha scritto:

Thank you very much for the data. However, the file (sst.nc) you sent had 1D coordinates. These kinds of coordinates are okay in GeoServer:

lat_1D: float32 lat(y)
lon_1D: float32 lon(x)

The latitudes and longitudes in a 2D NetCDF data have 2D shapes, and they depend on both the x and y axis. They are problematic in GeoServer, although in the documentations it is mentioned that they can be handled):

lat_2D: float32 lat(x,y)
lon_2D: float32 lon(x,y)

On Thu, May 4, 2023 at 5:09 PM Andrea Aime <andrea.aime@anonymised.com> wrote:

On Thu, May 4, 2023 at 3:09 PM Amirhossein Nikfal <ah.nikfal@anonymised.com> wrote:

Well actually in the documentation it has been mentioned that GeoServer can handle NetCDF data with 2D coordinates (latitudes/longitudes).

A scalar variable with lat and lon? Sure, it’s supported.
When you said a 2D variable, I understood you wanted a vector-type variable (not sure if NetCDF even supports it, TBH).

I just want a sample NetCDF data with 2D coordinates to test it.

This tiny NetCDF used for the NetCDF module tests should do the trick:
https://github.com/geoserver/geoserver/raw/main/src/extension/netcdf/src/test/resources/org/geoserver/netcdf/test-data/sst.nc

Regards
Andrea

==

GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

9

Okay, got it. However:

Quoting the documentations:

Two-dimension coordinate variables are exposed in GeoServer as single dimensions. Their domain is exposed in GetCapabilities as a flat list of possible values.

Should I expose their domain by myself? So they can be exposed as single dimension coordinates.

Best,
Amir

On Fri, May 5, 2023 at 3:28 PM Amirhossein Nikfal <ah.nikfal@anonymised.com> wrote:

I think it is clearly mentioned in the documentation that NetCDF with 2D lat/lon are supported by GeoServer:
https://docs.geoserver.org/stable/en/user/extensions/netcdf/netcdf.html#two-dimensional-coordinate-variables

I am looking for a sample NetCDF datafile with 2D coordinates (mentioned in documentation) to play with in GeoServer. My own datafiles (with 2D lat/lon) cannot be read by GeoServer.

Best,
Amir

On Fri, May 5, 2023 at 3:06 PM Andrea Aime <andrea.aime@anonymised.com> wrote:

Ah, it’s an irregular grid then, each entry of the array is mapped to its own independent point? No, as far as I know GeoServer has no support for it, regardless of the format (nothing specific to NetCDF). Where did you read those are supported?

I has trouble understanding what you meant so far, it’s likely there is a similar misunderstanding in the docs.

Andrea

Il ven 5 mag 2023, 13:57 Amirhossein Nikfal <ah.nikfal@anonymised.com> ha scritto:

Thank you very much for the data. However, the file (sst.nc) you sent had 1D coordinates. These kinds of coordinates are okay in GeoServer:

lat_1D: float32 lat(y)
lon_1D: float32 lon(x)

The latitudes and longitudes in a 2D NetCDF data have 2D shapes, and they depend on both the x and y axis. They are problematic in GeoServer, although in the documentations it is mentioned that they can be handled):

lat_2D: float32 lat(x,y)
lon_2D: float32 lon(x,y)

On Thu, May 4, 2023 at 5:09 PM Andrea Aime <andrea.aime@anonymised.com9…> wrote:

On Thu, May 4, 2023 at 3:09 PM Amirhossein Nikfal <ah.nikfal@anonymised.com> wrote:

Well actually in the documentation it has been mentioned that GeoServer can handle NetCDF data with 2D coordinates (latitudes/longitudes).

A scalar variable with lat and lon? Sure, it’s supported.
When you said a 2D variable, I understood you wanted a vector-type variable (not sure if NetCDF even supports it, TBH).

I just want a sample NetCDF data with 2D coordinates to test it.

This tiny NetCDF used for the NetCDF module tests should do the trick:
https://github.com/geoserver/geoserver/raw/main/src/extension/netcdf/src/test/resources/org/geoserver/netcdf/test-data/sst.nc

Regards
Andrea

==

GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail