[GRASS-dev] [GRASS GIS] #335: export floats and doubles with correct precision

GDAL · October 14, 2008, 5:35am

#335: export floats and doubles with correct precision
-------------------------+--------------------------------------------------
Reporter: hamish | Owner: grass-dev@lists.osgeo.org
Type: task | Status: new
Priority: minor | Milestone: 6.4.0
Component: default | Version: unspecified
Keywords: precision | Platform: Unspecified
Cpu: Unspecified |
-------------------------+--------------------------------------------------
see http://thread.gmane.org/gmane.comp.gis.grass.devel/29622/

this started with lib/db/dbmi_base/valuefmt.c %lf -> %.15g

High priority:
r.what, r.univar sum_str, r.info's fancy report min= max=, and r.colors.
Also 'g.region -g' -> lib/gis/wind_format.c:

{{{
static int format_double(double value, char *buf)
{
sprintf(buf, "%.8f", value);
G_trim_decimal(buf);

return 0;
}
}}}

I notice that raster/r.mapcalc/expression.c uses %.8g for a double.

for lat/lon, %.8f is approx 1mm (1852*60*1e-8; ie better than RTK GPS),
for meter/feet based units it's very very small.

I guess we may as well to do this properly, i.e. split off FCELL values to
something less precise ( %.6g, %f, ? ).
Maybe new G_format_double() and G_format_float() fns for that?

{{{
$ svngrep -r '%\.[1-9][0-9][fg]' * | cut -f1 -d: | uniq

display/d.rast.edit/edit.c
display/d.zoom/set.c
general/g.setproj/main.c
general/g.setproj/get_num.c
general/g.setproj/get_deg.c
lib/proj/convert.c
lib/proj/get_proj.c
lib/vector/rtree/gammavol.c
lib/vector/Vlib/intersect.c
lib/gis/quant_io.c
lib/gis/gislib.dox
lib/gis/proj3.c
lib/gis/color_write.c
lib/gis/cats.c
lib/g3d/g3dkeys.c
lib/g3d/writeascii.c
lib/g3d/filecompare.c
lib/g3d/g3dcats.c
lib/db/dbmi_base/datetime.c
lib/db/dbmi_base/valuefmt.c
ps/ps.map/ps_fclrtbl.c
raster/r.colors/rules.c
raster/r.stats/raw_stats.c
raster/r.univar2/stats.c
raster/r.reclass/main.c
raster/r.recode/read_rules.c
raster/r.info/main.c
raster/r.quant/read_rules.c
raster/r.distance/report.c
raster/r.cats/main.c
raster/r.external/main.c
raster/r.what/main.c
raster/r.statistics/o_average.c
raster/r.average/main.c
scripts/r.in.srtm/r.in.srtm
vector/v.to.points/main.c
vector/v.segment/main.c
vector/v.label.sa/main.c
vector/v.what.rast/main.c
vector/v.to.db/update.c
vector/v.to.db/report.c
vector/v.kernel/main.c
vector/v.in.ascii/points.c
}}}

the list is long and manual review is needed for each one.

Hamish

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/335>
GRASS GIS <http://grass.osgeo.org>

GDAL · October 15, 2008, 9:38am

#335: export floats and doubles with correct precision
--------------------------+-------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: task | Status: new
  Priority: minor | Milestone: 6.4.0
Component: default | Version: unspecified
Resolution: | Keywords: precision
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by neteler):

IMHO this needs to be fixed before 6.4.0 is released (hopefully end of the
year with release candidates starting as soon as vector undo is
implemented in the new digitizer).

Markus

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/335#comment:1>
GRASS GIS <http://grass.osgeo.org>

GDAL · November 11, 2008, 9:46am

#335: export floats and doubles with correct precision
--------------------------+-------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: task | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: unspecified
Resolution: | Keywords: precision
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Changes (by msieczka):

* priority: minor => critical

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/335#comment:2>
GRASS GIS <http://grass.osgeo.org>

GDAL · December 6, 2008, 2:29am

#335: export floats and doubles with correct precision
--------------------------+-------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: task | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: unspecified
Resolution: | Keywords: precision
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by hamish):

v.univar too,

{{{
g.copy vect=archsites,tmp_arch
v.db.addcol tmp_arch column='x double, y double'
v.to.db tmp_arch option=coor columns=x,y

eval `v.univar -g tmp_arch column=x type=point | grep -w mean`
X_MEAN=$mean

eval `v.univar -g tmp_arch column=y type=point | grep -w mean`
Y_MEAN=$mean

echo "Center of point cloud: $X_MEAN, $Y_MEAN"
}}}

Center of point cloud: 598326, 4.92146e+06

would be nice for y coord to be intact (ie preserve 3 digits to the right
of the decimal point (mm))

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:3>
GRASS GIS <http://grass.osgeo.org>

GDAL · June 4, 2009, 2:25am

#335: export floats and doubles with correct precision
--------------------------+-------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: task | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: unspecified
Resolution: | Keywords: precision
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by hamish):

'g.region -g' for lat/lon fixed for G_format_*() in r37725,6,7.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:4>
GRASS GIS <http://grass.osgeo.org>

GDAL · June 20, 2009, 1:47am

#335: export floats and doubles with correct precision
--------------------------+-------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: task | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: unspecified
Resolution: | Keywords: precision
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by hamish):

Replying to [comment:4 hamish]:
> 'g.region -g' for lat/lon fixed for G_format_*() in r37725,6,7.

(see also #654)

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:5>
GRASS GIS <http://grass.osgeo.org>

GDAL · August 10, 2009, 3:56am

#335: export floats and doubles with correct precision
--------------------------+-------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: task | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: unspecified
Resolution: | Keywords: precision
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by hamish):

Hi,

is there any reason not to change the following to %.15g?
{{{
raster/r.colors/
rules.c:47: sprintf(minstr, "%.25f", (double)min);
rules.c:48: sprintf(maxstr, "%.25f", (double)max);
}}}
AFAIK %.25f for a double (DCELL) is just reporting noise. It causes out-
of-range white areas on a raster map because the 0%,100% color rules are
no longer == the actual floating point value (50% chance it falls on the
outside of the range)

also, in lib/gis/color_write.c:
{{{
#define PRECISION 30
#define THRESHOLD .0000000000000000000000000000005
/* .5 * 10 ^(-30) */
...
static int format_min(char *str, double dval)
{
double dtmp;

     sprintf(str, "%.*f", PRECISION, dval);
     G_trim_decimal(str);
     sscanf(str, "%lf", &dtmp);
     if (dtmp != dval) { /* if no zeros after decimal point were
trimmed */
         sprintf(str, "%.*f", PRECISION, dval - THRESHOLD);
         /* because precision is probably higher than PRECISION */
     }

return 0;
}
}}}

(max is similar but adds the spillover threshold; but apparently this
doesn't have the desired effect as 5e-31 is swept away in the noise
anyway. ?)

I blindly guess number originally came from an attempt to be a wee bit
bigger than 1e-32. ??? (dates back to CVS rev1.1 so no help
from the commit history)

i.e. can we always assume DCELL|double to be the same bit depth, and %.15g
the most we'll ever be able to squeeze from it?

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:6>
GRASS GIS <http://grass.osgeo.org>

GDAL · August 10, 2009, 3:59am

#335: export floats and doubles with correct precision
--------------------------+-------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: task | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: unspecified
Resolution: | Keywords: precision
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by hamish):

oblig "What Every Computer Scientist Should Know About Floating-Point
Arithmetic", by David Goldberg:
http://docs.sun.com/source/806-3568/ncg_goldberg.html

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:7>
GRASS GIS <http://grass.osgeo.org>

GDAL · August 16, 2009, 9:00am

#335: export floats and doubles with correct precision
--------------------------+-------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: task | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: unspecified
Resolution: | Keywords: precision
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by hamish):

on grass-dev Glynn wrote:
> We can assume that it always refers to IEEE 754 double
> precision format. Systems where this isn't the case are
> extremely uncommon, and will almost certainly fail to
> support GRASS without a great deal of additional effort.

> And regardless of the precision of "double", I seriously
> doubt that anyone has actual data which is accurate to 15
> significant digits (equivalent to specifying the circumference
> of the earth to within 40 nanometres, or the height of Everest
> to within 9 picometres).

r.colors fixed in trunk with r38750, r38752.
(see also #720)

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:8>
GRASS GIS <http://grass.osgeo.org>

GDAL · August 20, 2009, 6:22am

#335: export floats and doubles with correct precision
--------------------------+-------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: task | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: unspecified
Resolution: | Keywords: precision
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by hamish):

r.info's data range (both fancy and shell style) updated in devbr6 and
trunk to use %.7g for floats and %.15g for doubles.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:9>
GRASS GIS <http://grass.osgeo.org>

GDAL · August 20, 2009, 7:43am

#335: export floats and doubles with correct precision
--------------------------+-------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: task | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: unspecified
Resolution: | Keywords: precision
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by hamish):

r.what's cell data output updated in devbr6 and trunk to use %.7g for
floats and %.15g for doubles.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:10>
GRASS GIS <http://grass.osgeo.org>

GDAL · September 20, 2009, 5:21am

#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: task | Status: new
  Priority: critical | Milestone: 6.4.1
Component: default | Version: svn-develbranch6
Resolution: | Keywords: precision
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Changes (by hamish):

  * platform: Unspecified => All
  * version: unspecified => svn-develbranch6
  * cpu: Unspecified => All
  * milestone: 6.4.0 => 6.4.1

Comment:

good progress made on the critical cases, deferring the rest 'til 6.4.1.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:11>
GRASS GIS <http://grass.osgeo.org>

GDAL · January 7, 2010, 12:05pm

#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: task | Status: new
  Priority: critical | Milestone: 6.4.1
Component: default | Version: svn-develbranch6
Resolution: | Keywords: precision
  Platform: All | Cpu: All
-----------------------+----------------------------------------------------
Changes (by hamish):

* milestone: => 6.4.1

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:13>
GRASS GIS <http://grass.osgeo.org>

GDAL · January 27, 2011, 9:47pm

#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: task | Status: new
Priority: critical | Milestone: 6.4.1
Component: Default | Version: svn-develbranch6
Keywords: precision | Platform: All
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by hamish):

with respect to raster/r.stats/raw_stats.c (and so r.out.xyz), the
_is_fp() array-filling in main.c needs to be changed to fill the array
with the map type, instead of just categorical or FP. The array is already
type int, so this shouldn't be too hard. Then raw_stats.c can output %.7g
or %.15g depending on FCELL or DCELL (see r.info), instead of the current
%.10f for all.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:14>
GRASS GIS <http://grass.osgeo.org>

GDAL · January 31, 2011, 3:59am

#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: task | Status: new
Priority: critical | Milestone: 6.4.1
Component: Default | Version: svn-develbranch6
Keywords: precision | Platform: All
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by hamish):

r.what.color in the dev branches changed to %.15g for FP maps, but that
should be broken down by FCELL & DCELL, and FCELL should use %.7g.

Hamish

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/335#comment:15>
GRASS GIS <http://grass.osgeo.org>

GDAL · March 18, 2011, 2:13am

#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: task | Status: new
Priority: critical | Milestone: 6.4.1
Component: Default | Version: svn-develbranch6
Keywords: precision | Platform: All
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by hamish):

r.reclass done in trunk and devbr6 with r45696,7. Should be backported to
relbr64 after some period of testing.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:16>
GRASS GIS <http://grass.osgeo.org>

GDAL · September 23, 2011, 4:19am

#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: task | Status: new
Priority: critical | Milestone: 6.4.1
Component: Default | Version: svn-develbranch6
Keywords: precision | Platform: All
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by hamish):

something thing to keep in mind as we work through these: GRASS's
processing modules can be very(!) useful for processing imagery from
scanning electron microscopes. Typical pixel resolution for that might be
1 micron. aka the map units aren't necessarily in degrees, meters, or some
flavor of feet.

at the other end of things, for star catalog az,el,distance point data
projected from a lat/lon import the map units might be parsec or light-
years. (I suspect r.mapcalc et al. would also be highly useful for
processing large telescope images)

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:17>
GRASS GIS <http://grass.osgeo.org>

GDAL · June 23, 2012, 12:34pm

#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: task | Status: new
Priority: critical | Milestone: 6.4.1
Component: Default | Version: svn-develbranch6
Keywords: precision | Platform: All
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by hamish):

devbr6: the precision stored in colr/ files done in r52194, d.what.rast
done in r52195.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:18>
GRASS GIS <http://grass.osgeo.org>

GDAL · August 19, 2012, 8:22am

#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: task | Status: new
Priority: critical | Milestone: 6.4.1
Component: Default | Version: svn-develbranch6
Keywords: precision | Platform: All
Cpu: All |
-----------------------+----------------------------------------------------

Comment(by neteler):

Please backport to 6.4 since not much testing happens in 6.5 (but in
6.4.svn).
(I see that many backports are missing!).

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:19>
GRASS GIS <http://grass.osgeo.org>

GDAL · August 19, 2012, 8:26am

#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: task | Status: new
Priority: critical | Milestone: 6.4.3
Component: Default | Version: svn-develbranch6
Keywords: precision | Platform: All
Cpu: All |
-----------------------+----------------------------------------------------
Changes (by martinl):

* milestone: 6.4.1 => 6.4.3

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/335#comment:20>
GRASS GIS <http://grass.osgeo.org>