[GRASS-user] Problem running v.rast.stats - caused by calculating coefficient of variance when mean=0?

GRASS GRASS user
1

Hello,

I am trying to use v.rast.stats in GRASS 6.4, but cannot get it to work with
the vector file I am using. Vector data I am using is administrative
boundaries (provinces of Vietnam) with the raster being forest cover data at
500m x 500m pixel size.

Interestingly, it works fine when I use certain subsets of my vector file
(various sets of provinces selected at random), just not when I use the
whole file (i.e. all provinces). When I use the complete provinces file, it
finishes with errors (copied below). When I check the attribute table, it
has created the 9 columns it is supposed to, but they are all blank.

Because of the error message ("TrChng_cf_=inf WHERE cat=3", I think the
issue may have to do with calculating the coefficient of variation when the
mean of an area is equal to zero. The precision of this data is not
particularly high, and pixels with value zero are common, so with many small
islands, the likelihood of a couple of them having a mean of zero is quite
high.

If v.rast.stats (which performs its calculations by calling r.univar, I
believe) is unable to calculate the coefficient of variation in one category
because of a mean of zero, does that cause the rest of the run to be
unsuccessful? Is that what is happening here? If so, is there any way
around this? I am not even interested in the coefficient of variation, as I
am only looking for very basic stats (n, mean, max, min). Is there any way
to tell it to skip the CV, or is there another work-around anyone can think
of? Or is this some other problem entirely?

Any advice would be greatly appreciated.

Thanks,
Tim

GRASS 6.4.0RC4 (MODIS_TreeCover):~ > v.rast.stats vector=VN_province
raster=MODIS_VCF_TreeCoverChange_2000_2005_indochine colprefix=TrChng
Raster MASK found, temporarily disabled
WARNING: raster <MASK> not found
WARNING: Area without centroid (may be OK for island)
WARNING: Area without centroid (may be OK for island)
WARNING: Area without centroid (may be OK for island)
WARNING: Area without centroid (may be OK for island)
WARNING: Area without centroid (may be OK for island)
WARNING: Area without centroid (may be OK for island)
WARNING: Area without centroid (may be OK for island)
Adding columns < TrChng_n INTEGER, TrChng_min DOUBLE PRECISION, TrChng_max
DOUBLE PRECISION, TrChng_ran DOUBLE PRECISION, TrChng_mea DOUBLE PRECISION,
TrChng_std DOUBLE PRECISION, TrChng_var DOUBLE PRECISION, TrChng_cf_ DOUBLE
PRECISION, TrChng_sum DOUBLE PRECISION>
Processing data ...
Updating the database ...
ERROR: This should not happen: wrong return type in parsing.
dbmi: Protocol error
ERROR: Error while executing: 'UPDATE NorthEast_NoIslands SET
       TrChng_cf_=inf WHERE cat=3'
Restoring previous MASK...
WARNING: raster <63570.0_origmask> not found

--
View this message in context: http://n2.nabble.com/Problem-running-v-rast-stats-caused-by-calculating-coefficient-of-variance-when-mean-0-tp3502369p3502369.html
Sent from the Grass - Users mailing list archive at Nabble.com.

2

Tim wrote:

I am trying to use v.rast.stats in GRASS 6.4, but cannot get it to
work with the vector file I am using. Vector data I am using is
administrative boundaries (provinces of Vietnam) with the raster
being forest cover data at 500m x 500m pixel size.

Interestingly, it works fine when I use certain subsets of
my vector file (various sets of provinces selected at random), just not
when I use the whole file (i.e. all provinces). When I use the
complete provinces file, it finishes with errors (copied below).
When I check the attribute table, it has created the 9 columns it
is supposed to, but they are all blank.

....

Updating the database ...
ERROR: This should not happen: wrong return type in
parsing.
dbmi: Protocol error
ERROR: Error while executing: 'UPDATE NorthEast_NoIslands
       SET TrChng_cf_=inf WHERE cat=3'

....

Because of the error message ("TrChng_cf_=inf WHERE cat=3",
I think the issue may have to do with calculating the coefficient
of variation when the mean of an area is equal to zero.
The precision of this data is not particularly high, and pixels
with value zero are common, so with many small islands, the likelihood
of a couple of them having a mean of zero is quite high.

If v.rast.stats (which performs its calculations by calling r.univar,
I believe)

yes

is unable to calculate the coefficient of variation in one category
because of a mean of zero, does that cause the rest of the run to be
unsuccessful? Is that what is happening here?

#test
r.mapcalc zero=0
r.univar zero

sets cf_var to nan for me.

If I try to update a DBF table with a nan or inf into to a double column:
v.db.addcol local_bugs col='cfv double'
echo "UPDATE local_bugs SET cfv=nan" | db.execute
echo "UPDATE local_bugs SET cfv=inf" | db.execute

both those work for me as far as I can tell (v.db.select shows those
columns as empty)

If so, is there any way around this? I am not even interested in the
coefficient of variation, as I am only looking for very basic stats
(n, mean, max, min). Is there any way to tell it to skip the CV, or
is there another work-around anyone can think of?

to test your theory: find and edit the v.rast.stats script,
find this line:
  BASECOLS="n min max range mean stddev variance cf_var sum"
and remove cf_var from it.

Or is this some other problem entirely?

could be... but the "inf" is definitely an obvious suspect.

also you might try changing to the SQLite driver instead of the less
flexible DBF one. (see db.connect help page)

Hamish

3

Thanks Hamish. You were right about the CV returning 'nan' on zeros, and in
the end it did seem that was not the problem.

The issue eventually seemed to be that the column prefixes I was using were
too long. When I went down to using 2- and 3-letter prefixes, it seemed to
start working fine (initially I had been using 5- or 6-letter prefixes). I
probably should have thought of that earlier, but...

In any case, working well now, so thanks for the help.

Best,
Tim

hamish-2 wrote:

Tim wrote:

I am trying to use v.rast.stats in GRASS 6.4, but cannot get it to
work with the vector file I am using. Vector data I am using is
administrative boundaries (provinces of Vietnam) with the raster
being forest cover data at 500m x 500m pixel size.

Interestingly, it works fine when I use certain subsets of
my vector file (various sets of provinces selected at random), just not
when I use the whole file (i.e. all provinces). When I use the
complete provinces file, it finishes with errors (copied below).
When I check the attribute table, it has created the 9 columns it
is supposed to, but they are all blank.

....

Updating the database ...
ERROR: This should not happen: wrong return type in
parsing.
dbmi: Protocol error
ERROR: Error while executing: 'UPDATE NorthEast_NoIslands
       SET TrChng_cf_=inf WHERE cat=3'

....

Because of the error message ("TrChng_cf_=inf WHERE cat=3",
I think the issue may have to do with calculating the coefficient
of variation when the mean of an area is equal to zero.
The precision of this data is not particularly high, and pixels
with value zero are common, so with many small islands, the likelihood
of a couple of them having a mean of zero is quite high.

If v.rast.stats (which performs its calculations by calling r.univar,
I believe)

yes

is unable to calculate the coefficient of variation in one category
because of a mean of zero, does that cause the rest of the run to be
unsuccessful? Is that what is happening here?

#test
r.mapcalc zero=0
r.univar zero

sets cf_var to nan for me.

If I try to update a DBF table with a nan or inf into to a double column:
v.db.addcol local_bugs col='cfv double'
echo "UPDATE local_bugs SET cfv=nan" | db.execute
echo "UPDATE local_bugs SET cfv=inf" | db.execute

both those work for me as far as I can tell (v.db.select shows those
columns as empty)

If so, is there any way around this? I am not even interested in the
coefficient of variation, as I am only looking for very basic stats
(n, mean, max, min). Is there any way to tell it to skip the CV, or
is there another work-around anyone can think of?

to test your theory: find and edit the v.rast.stats script,
find this line:
  BASECOLS="n min max range mean stddev variance cf_var sum"
and remove cf_var from it.

Or is this some other problem entirely?

could be... but the "inf" is definitely an obvious suspect.

also you might try changing to the SQLite driver instead of the less
flexible DBF one. (see db.connect help page)

Hamish

_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

--
View this message in context: http://n2.nabble.com/Problem-running-v-rast-stats-caused-by-calculating-coefficient-of-variance-when-mean-0-tp3502369p3521188.html
Sent from the Grass - Users mailing list archive at Nabble.com.

4

Tim wrote:

The issue eventually seemed to be that the column prefixes I was using
were too long. When I went down to using 2- and 3-letter prefixes, it
seemed to start working fine (initially I had been using 5- or
6-letter prefixes).

I've now submitted a check in 6.5svn that makes it exit with an error if
you give it more than 6 chars. DBF only allows 10 chars max, and [_mea]n
and [_med]ian need 4 of those to stay unique. (The python version in gr7
silently chops away anything more)

Suggestion: switch to SQLite as the DB backend. See the db.connect help
page.

Hamish