which rather should be
3266766176256 - 241981407231 = 3024784769025
It seems to me that the subtraction is working fine for small numbers but not for larger numbers.
I tried all the combinations of
r.mapcalc " output = float( float(input) - $min ) "
r.mapcalc " output = float( input - $min ) "
Attached the script.
Did I reach some numerical limitation?
Any suggestions?
Thanks giuseppe
Hello Giuseppe,
I am too lazy to calculate binary representation of your numbers, but
your suspicion of insufficient precision of float seems to be spot-on.
If output is set to DCELL, then the result is exactly as you would
expect. Try yourself:
r.mapcalc expression="output=basin - 241981407231.0"
Range of data: min = 1 max = 3024784769025
Markus, output of r.univar and r.info both can have scientific
notation in their range outputs.
Giuseppe, at least on my system r.mapcalc for floating point
expressions (if one of operands is floating point) defaults to double
(stored as DCELL). Still to be safe you can convert one of operands
(e.g. input map with ints) to double with double() as Markus already
pointed out.
Thanks to both of you!!
Now I got it! So the
Float32, CFloat32 -3.4E38 3.4E38 range identify the maximum number of characters in the binary format and not in the decimal format - correct?
So even if 3024784769025 fit as a decimal number in -3.4E38 3.4E38 does not fit as binary - correct?
…this one I did not know… I spent a few days on this.
yes r.info gives the scientific format so it will be nice to have also the long decimal number… .to spot this kind of errors.
Giuseppe
···
Giuseppe Amatulli, Ph.D.
Research scientist at
School of the Environment
Yale University
New Haven, CT, USA - 06511
Sorry, my wrong conclusion
Single-precision floating-point variables are able to represent integers between [-16777216, 16777216]
maybe better to report in the r.out.gdal data type table
···
Giuseppe Amatulli, Ph.D.
Research scientist at
School of the Environment
Yale University
New Haven, CT, USA - 06511
On Fri, Feb 26, 2021 at 2:20 AM Giuseppe Amatulli
<giuseppe.amatulli@gmail.com> wrote:
Sorry, my wrong conclusion
Single-precision floating-point variables are able to represent integers between [-16777216, 16777216]
maybe better to report in the r.out.gdal data type table
FWIW: We have some info about raster precision and semantics here:
I think an overview table, like https://gitlab.com/-/snippets/2083412, should be part of both the raster introduction text and the manual of r.mapcalc.
I think an overview table, like https://gitlab.com/-/snippets/2083412, should be part of both the raster introduction text and the manual of r.mapcalc.
