[GRASS-dev] [GRASS GIS] #420: r.quantile returns wrong values

GRASS GRASS developer
1

#420: r.quantile returns wrong values
------------------------+---------------------------------------------------
Reporter: jarekj71 | Owner: grass-dev@lists.osgeo.org
     Type: defect | Status: new
Priority: minor | Milestone: 6.4.0
Component: default | Version: svn-trunk
Keywords: r.quantile | Platform: Linux
      Cpu: x86-32 |
------------------------+---------------------------------------------------
For spearfish database:
g.region res=30

r.info -r -s map=elevation.dem@PERMANENT
min=1066
max=1840
nsres=30
ewres=30

r.quantile input=elevation.dem@PERMANENT
percentiles=0.001,0.01,0.1,0.25,0.50,0.75,.90,0.99,0.999 bins=1000000
returns:
0:0.001000:1067.000000
1:0.010000:1070.000000
2:0.100000:1080.000000
3:0.250000:1086.000000
4:0.500000:1091.000000
5:0.750000:1095.000000
6:0.900000:1098.000000
7:0.990000:1099.000000
8:0.999000:1099.000000

While equivalent command in R:

library(spgrass6)
readRAST6("elevation.dem", plugin=FALSE, useGDAL=TRUE)

> summary(r@data$elevation.dem)
    Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
    1066 1200 1316 1354 1488 1840 3294

quantile(na.omit(r@data$elevation.dem),c(0.001,0.01,0.1,0.25,0.50,0.75,.90,0.99,0.999))

returns:

     0.1% 1% 10% 25% 50% 75% 90% 99%
1080.000 1099.000 1156.000 1200.000 1316.000 1488.000 1621.000 1745.000
    99.9%
1815.684

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/420&gt;
GRASS GIS <http://grass.osgeo.org>

2

#420: r.quantile returns wrong values
-----------------------+----------------------------------------------------
  Reporter: jarekj71 | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: closed
  Priority: minor | Milestone: 6.4.0
Component: default | Version: svn-trunk
Resolution: invalid | Keywords: r.quantile
  Platform: Linux | Cpu: x86-32
-----------------------+----------------------------------------------------
Changes (by glynn):

  * status: new => closed
  * resolution: => invalid

Comment:

Replying to [ticket:420 jarekj71]:
> For spearfish database:
> g.region res=30

> r.quantile input=elevation.dem@PERMANENT
> percentiles=0.001,0.01,0.1,0.25,0.50,0.75,.90,0.99,0.999 bins=1000000
> returns:

The arguments to the percentiles= option are, well, percentiles, i.e. the
values given are divided by 100 to get fractions. So 0.99 is 0.99% not
99%.

Using correct values gives results which essentially agree with R:
{{{
$ r.quantile input=elevation.dem@PERMANENT
percentiles=0.1,1,10,25,50,75,90,99,99.9 bins=1000000
Computing histogram
Computing bins
Binning data
Sorting bins
Computing quantiles
0:0.100000:1080.000000
1:1.000000:1099.000000
2:10.000000:1156.000000
3:25.000000:1200.000000
4:50.000000:1316.000000
5:75.000000:1488.000000
6:90.000000:1621.000000
7:99.000000:1745.000000
8:99.900000:1816.000000
}}}

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/420#comment:1&gt;
GRASS GIS <http://grass.osgeo.org>