[GRASS-user] r.univar: total null and non-null cells: -1580967296

GRASS GRASS user
1

Hi,

r.univar gives me negative cell counts, whereas r.stats -c produces correct results (I hope ;-):

(...)
41 118441
42 193
43 135
44 8189
* 2704177984

I guess this is the same integer overflow issue reported earlier [1]. I am just a bit puzzled that after changing to a 64-bit Linux OS, results of r.stats seem to be correct, but not those of r.univar.

Am I missing something, perhaps an upgrade? (I'm using GRASS 6.4.0.)

Hermann

[1] http://lists.osgeo.org/pipermail/grass-user/2009-October/052574.html

2

On Tue, Aug 23, 2011 at 8:48 AM, Hermann Peifer <peifer@gmx.eu> wrote:

Hi,

r.univar gives me negative cell counts, whereas r.stats -c produces correct
results (I hope ;-):

(...)
41 118441
42 193
43 135
44 8189
* 2704177984

I guess this is the same integer overflow issue reported earlier [1]. I am
just a bit puzzled that after changing to a 64-bit Linux OS, results of
r.stats seem to be correct, but not those of r.univar.

Am I missing something, perhaps an upgrade? (I'm using GRASS 6.4.0.)

Hermann

[1] http://lists.osgeo.org/pipermail/grass-user/2009-October/052574.html

I can confirm this (64bit Linux):

GRASS 6.4.2svn (nc_spm_08):~ > g.region -dg res=8
n=320000
s=10000
w=120000
e=935000
nsres=8
ewres=8
rows=38750
cols=101875
cells=3947656250

GRASS 6.4.2svn (nc_spm_08):~ > r.univar elevation
100%
total null and non-null cells: -347311046 <<--- !
total null cells: -350476046 <<--- !

Of the non-null cells:
----------------------
n: 3165000
minimum: 55.5788
maximum: 156.33
range: 100.751
mean: 110.371
mean of absolute values: 110.371
standard deviation: 20.3162
variance: 412.748
variation coefficient: 18.4071 %
sum: 349325348.8554420471

Same in GRASS 7:

GRASS 7.0.svn (nc_spm_08@neteler):~ > r.univar elevation
100%
total null and non-null cells: -347311046
total null cells: -350476046

Of the non-null cells:
----------------------
n: 3165000
minimum: 55.5788
maximum: 156.33
range: 100.751
mean: 110.371
mean of absolute values: 110.371
standard deviation: 20.3162
variance: 412.748
variation coefficient: 18.4071 %
sum: 349325348.8554420471

I suppose that r.univar is lacking the needed off_t updates.

Markus

3

Markus wrote:

I can confirm this (64bit Linux):

GRASS 6.4.2svn (nc_spm_08):~ > g.region -dg res=8

...

GRASS 6.4.2svn (nc_spm_08):~ > r.univar elevation
100%
total null and non-null cells: -347311046
total null cells: -350476046

...

I suppose that r.univar is lacking the needed off_t updates.

globals.h is defining univar_stat's 'n' and 'size' as int, and
stats.c is printing as plain '%d'. (fwiw in GRASS 5 I used
'unsigned int' originally, but it appears to haven be lost with
the introduction of r3.univar.)

suggest to use 'unsigned long' and "%lu" as is done by r.in.xyz,
and 'off_t' for 'n_alloc'.

try r48240 in trunk.

Hamish

ps- I get this warning when compiling it, not sure how to fix:
r3.univar_main.c: In function 'main':
r3.univar_main.c:146: warning: assignment discards qualifiers from pointer target type
146: if (NULL == (mapset = G_find_raster3d(zonemap, "")))

4

On 11/09/2011 23:26, Hamish wrote:

try r48240 in trunk.

I downloaded grass7 from trunk and its r.univar now reports for the same map I used earlier:

> total null and non-null cells: 2714000000 (rather than -1580967296)

Thanks for the fix. Will it also find its way into version 6.4.2?

Hermann

5

Hamish wrote:

suggest to use 'unsigned long' and "%lu" as is done by r.in.xyz,
and 'off_t' for 'n_alloc'.

There is no point in using anything larger than size_t for n_alloc.

--
Glynn Clements <glynn@gclements.plus.com>