Hi,
I have the following raster map:
Type of Map: raster Number of Categories: 255 Data Type: CELL Rows: 58000 Columns: 67000 Total Cells: 3886000000
r.stats (-c and -p) tells me:
1 608232 1 -0.15%
2 13691259 2 -3.35%
3 2047970 3 -0.50%
4 194288 4 -0.05%
5 105156 5 -0.03%
6 303014 6 -0.07%
7 632953 7 -0.15%
8 110266 8 -0.03%
9 119532 9 -0.03%
10 301250 10 -0.07%
11 859077 11 -0.21%
12 110022085 12 -26.90%
13 3174787 13 -0.78%
14 587198 14 -0.14%
15 3958146 15 -0.97%
16 2526227 16 -0.62%
17 3996373 17 -0.98%
18 38131042 18 -9.32%
19 989760 19 -0.24%
20 27504344 20 -6.73%
21 22922649 21 -5.61%
22 3188788 22 -0.78%
23 52637630 23 -12.87%
24 71417325 24 -17.46%
25 29867401 25 -7.30%
26 12513397 26 -3.06%
27 17347949 27 -4.24%
28 10163292 28 -2.49%
29 25264490 29 -6.18%
30 710832 30 -0.17%
31 6661714 31 -1.63%
32 13529887 32 -3.31%
33 150968 33 -0.04%
34 1583419 34 -0.39%
35 1275927 35 -0.31%
36 9942214 36 -2.43%
37 314126 37 -0.08%
38 75737 38 -0.02%
39 1141921 39 -0.28%
40 1231039 40 -0.30%
41 11439484 41 -2.80%
42 590138 42 -0.14%
43 331796 43 -0.08%
255 -913132378 255 223.28%
I guess there is an integer overflow somewhere(?)
Hermann
Hamish
2
which versions of grass? the more detailed the answer the
better, I am wondering if it is before or after a recent
off-by-one fix.
tx,
Hamish
--- On Thu, 1/10/09, Hermann Peifer <peifer@gmx.eu> wrote:
From: Hermann Peifer <peifer@gmx.eu>
Subject: [GRASS-user] r.stats: negative cell counts and percentages
To: grass-user@lists.osgeo.org
Received: Thursday, 1 October, 2009, 10:54 PM
Hi,
I have the following raster map:
Type of Map: raster
Number of Categories: 255
Data Type: CELL Rows:
58000
Columns: 67000
Total Cells: 3886000000
r.stats (-c and -p) tells me:
1 608232 1
-0.15%
2 13691259 2
-3.35%
3 2047970
3 -0.50%
4 194288 4
-0.05%
5 105156 5
-0.03%
6 303014 6
-0.07%
7 632953 7
-0.15%
8 110266 8
-0.03%
9 119532 9
-0.03%
10 301250
10 -0.07%
11 859077
11 -0.21%
12 110022085 12 -26.90%
13 3174787 13
-0.78%
14 587198
14 -0.14%
15 3958146 15
-0.97%
16 2526227 16
-0.62%
17 3996373 17
-0.98%
18 38131042 18
-9.32%
19 989760
19 -0.24%
20 27504344 20
-6.73%
21 22922649 21
-5.61%
22 3188788 22
-0.78%
23 52637630 23
-12.87%
24 71417325 24
-17.46%
25 29867401 25
-7.30%
26 12513397 26
-3.06%
27 17347949 27
-4.24%
28 10163292 28
-2.49%
29 25264490 29
-6.18%
30 710832
30 -0.17%
31 6661714 31
-1.63%
32 13529887 32
-3.31%
33 150968
33 -0.04%
34 1583419 34
-0.39%
35 1275927 35
-0.31%
36 9942214 36
-2.43%
37 314126
37 -0.08%
38 75737 38
-0.02%
39 1141921 39
-0.28%
40 1231039 40
-0.30%
41 11439484 41
-2.80%
42 590138
42 -0.14%
43 331796
43 -0.08%
255 -913132378 255 223.28%
I guess there is an integer overflow somewhere(?)
Hermann
_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user
svn up https://svn.osgeo.org/grass/grass/branches/releasebranch_6_4 grass64_release
(on 8 Sptember 2009)
Hermann
Hamish wrote:
which versions of grass? the more detailed the answer the
better, I am wondering if it is before or after a recent
off-by-one fix.
tx,
Hamish
--- On Thu, 1/10/09, Hermann Peifer <peifer@gmx.eu> wrote:
From: Hermann Peifer <peifer@gmx.eu>
Subject: [GRASS-user] r.stats: negative cell counts and percentages
To: grass-user@lists.osgeo.org
Received: Thursday, 1 October, 2009, 10:54 PM
Hi,
I have the following raster map:
Type of Map: raster Number of Categories: 255 Data Type: CELL Rows: 58000 Columns: 67000 Total Cells: 3886000000
r.stats (-c and -p) tells me:
1 608232 1 -0.15%
2 13691259 2 -3.35%
3 2047970 3 -0.50%
4 194288 4 -0.05%
5 105156 5 -0.03%
6 303014 6 -0.07%
7 632953 7 -0.15%
8 110266 8 -0.03%
9 119532 9 -0.03%
10 301250 10 -0.07%
11 859077 11 -0.21%
12 110022085 12 -26.90%
13 3174787 13 -0.78%
14 587198 14 -0.14%
15 3958146 15 -0.97%
16 2526227 16 -0.62%
17 3996373 17 -0.98%
18 38131042 18 -9.32%
19 989760 19 -0.24%
20 27504344 20 -6.73%
21 22922649 21 -5.61%
22 3188788 22 -0.78%
23 52637630 23
-12.87%
24 71417325 24
-17.46%
25 29867401 25 -7.30%
26 12513397 26 -3.06%
27 17347949 27 -4.24%
28 10163292 28 -2.49%
29 25264490 29 -6.18%
30 710832 30 -0.17%
31 6661714 31 -1.63%
32 13529887 32 -3.31%
33 150968 33 -0.04%
34 1583419 34 -0.39%
35 1275927 35 -0.31%
36 9942214 36 -2.43%
37 314126 37 -0.08%
38 75737 38 -0.02%
39 1141921 39 -0.28%
40 1231039 40 -0.30%
41 11439484 41 -2.80%
42 590138 42 -0.14%
43 331796 43 -0.08%
255 -913132378 255 223.28%
I guess there is an integer overflow somewhere(?)
Hermann
_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user