[Geoserver-devel] [JIRA] (GEOS-8632) Incorrect packing formula used in NetCDF scalefactor and add_offset calculations

GeoServer GeoServer Developer
1

Jerry Wilwerding created an issue

GeoServer / BugGEOS-8632

Incorrect packing formula used in NetCDF scalefactor and add_offset calculations

Issue Type:

BugBug

Affects Versions:

2.12.2

Assignee:

Unassigned

Created:

14/Mar/18 12:55 PM

Priority:

MediumMedium

Reporter:

Jerry Wilwerding

I am using geoserver v2.12.2 with the JVM system property to have geoserver
unpack NetCDF data with respect to the scale_factor/add_offset attributes:
-Dorg.geotools.coverage.io.netcdf.enhance.ScaleMissing=true
Geoserver seems to be doing the unpacking fine, but when I configure my
coverage.xml to re-pack the data into a smaller datatype (such as byte) the
formula used treats the datatype as unsigned and when the NetCDF file is
read by external software the values are interpreted incorrectly since
NetCDF convention defines the packed data types (Byte, Short, Int) as signed
integers. Given this convention, it appears that the incorrect packing algorithm was chosen resulting in this bug.

I would recommend that the formula be updated to use the “signed”
one below until “ubyte, ushort, and uint” data types are supported by
geoserver for packing.

If changing the formula is not an option, then the one really should describe the data type as unsigned since using unsigned datatypes is contrary to netCDF conventions.

Impact to this bug is that any data value in the upper half of the range will be interpreted as a negative value by most clients since data types are signed by convention.

For reference, the formulas are defined in DataPacking.java
https://github.com/geoserver/geoserver/blob/master/src/extension/netcdf-out/src/main/java/org/geoserver/web/netcdf/DataPacking.java

There is more information at the link below, but these seem to be the two
recommended options for data packing while reserving space for missing data.
The first is for signed packed values and the second is for unsigned packed
values which is the one that geoserver is using.
https://www.unidata.ucar.edu/software/netcdf/docs/BestPractices.html#Packed%20Data%20Values

In either the signed or unsigned case, an alternate formula may be used for
the add_offset and scale_factor packing parameters that reserves a packed
value for a special value, such as an indicator of missing data. For
example, to reserve the minimum packed value (-2^n\ -\ 1^) for use as a
special value in the case of signed packed values:

> scale_factor =(dataMax - dataMin) / (2^n^ - 2)

> add_offset = (dataMax + dataMin) / 2

If the packed values are unsigned, then the analogous formula that reserves
0 as the packed form of a special value would be:

> scale_factor =(dataMax - dataMin) / (2^n^ - 2)

> add_offset = dataMin - scale_factor

Add Comment

Add Comment

Get Jira notifications on your phone! Download the Jira Cloud app for Android or iOS

This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100082-sha1:b581f55)

Atlassian logo