[Geoserver-devel] force decimals

Hi,

I was looking at the output that was generated with forcedDecimals for WFS GetFeature MapML formatted requests, and I noticed that for Web Mercator responses, large negative decimal coordinate values were ignoring the forcedDecimal setting from the user interface, returning the coordinate in scientific notation. I traced this back to the GeoTools CoordinateFormatter class.

GeoTools’ CoordinateFormatter has a condition that doesn’t use forcedDecimal when the supplied number is in the -DECIMAL_MIN to +DECIMAL_MIN range, or when the value of the supplied number is less than the DECIMAL_MAX field, or zero. I believe the intent was that the default formatting code path (scientific notation, presumably) should be used when the value of the supplied number was between -DECIMAL_MAX and +DECIMAL_MAX but not in the -DECIMAL_MIN to +DECIMAL_MIN range, nor zero.

So I changed it, and added some tests to cover the changes, and without changing any other tests, it passes. I also ran the GeoServer tests on these branches / artifacts and they also pass.

If I don’t hear differing opinions in response to this, I’ll go ahead and create a JIRA issue and a couple of PRs to close the issue.

Thanks!
Peter

Hi Peter,
turns out I wrote that code… but too many years ago to remember :smiley:
So I can’t quite recall the original intention, but I believe you got it right,
the fast path with the truncation should be used also for numbers that are not too big in absolute terms, that is, going from
“x < DECIMAL_MAX” to “abs(x) < DECIMAL_MAX” looks indeed like an improvement

Cheers
Andrea

On Fri, Aug 13, 2021 at 10:57 PM Peter Rushforth <peter.rushforth@anonymised.com> wrote:

Hi,

I was looking at the output that was generated with forcedDecimals for WFS GetFeature MapML formatted requests, and I noticed that for Web Mercator responses, large negative decimal coordinate values were ignoring the forcedDecimal setting from the user interface, returning the coordinate in scientific notation. I traced this back to the GeoTools CoordinateFormatter class.

GeoTools’ CoordinateFormatter has a condition that doesn’t use forcedDecimal when the supplied number is in the -DECIMAL_MIN to +DECIMAL_MIN range, or when the value of the supplied number is less than the DECIMAL_MAX field, or zero. I believe the intent was that the default formatting code path (scientific notation, presumably) should be used when the value of the supplied number was between -DECIMAL_MAX and +DECIMAL_MAX but not in the -DECIMAL_MIN to +DECIMAL_MIN range, nor zero.

So I changed it, and added some tests to cover the changes, and without changing any other tests, it passes. I also ran the GeoServer tests on these branches / artifacts and they also pass.

If I don’t hear differing opinions in response to this, I’ll go ahead and create a JIRA issue and a couple of PRs to close the issue.

Thanks!
Peter


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel