[GRASS-user] r.kappa query

GRASS GRASS user
1

Hi list,

I have a query regarding the error matrix produced
using r.kappa (in GRASS 6.4.0 RC6) and hoping that
the following will help someone to replicate my findings.
From what I see, the error matrix is mis-labeling the
rows/columns in the matrix.

Using the spearfish dataset, I digitised 18 points
to create a reference dataset to validate
the classification, in this case the raster ‘landcover.orig’
For the purposes of the test and simplicity, I ensured
that the points were located within two landcover classes
(raster values 42 and 81). The following steps were carried out
to generate the output from r.kappa

#Create reference dataset
d.where | r.what input=landcover.orig > kappa_test.txt

For the purposes of this test, I deliberately changed the
landcover class from 42 to 32 in the validation dataset [1]
This will create an off-diagonal value in the error matrix.

#Create the vector point from ascii and convert to raster
v.in.ascii input=kappa_test output=kapp_test cat=4
v.to.rast input=kapp_test output=validation use=cat

#Run r.kappa
r.kappa classification=landcover.orig reference=validation output=validation.txt

The output from r.kappa (validation.txt) is below, however, it appears
to me that the matrix is transposed, i.e. the horizontal values
(MAP1) should in fact be vertical. From my interpretation, the error
matrix indicates that 8 validation points with the value 42 were
classified as 32, but in fact it should state that 8 points with
the value of 32 were classified as 42? Can anyone confirm
my findings?

Thanks in advance.

Regards,
Daniel.

ACCURACY ASSESSMENT
LOCATION: spearfish60 Fri Apr 1 12:45:42 2011
MASK: none
MAPS: MAP1 = Categories (validation in user1)
MAP2 = (untitled) (landcover.orig in PERMANENT)

Error Matrix
Panel #1 of 1
MAP1
cat# 32 42 81 Row Sum
M 32 0 8 0 8
A 42 0 0 0 0
P 81 0 0 10 10
Col Sum 0 8 10 18

Cats % Commission % Ommission Estimated Kappa
32 100.000000 -nan -999.000000
42 NA NA NA
81 0.000000 0.000000 1.000000

Kappa Kappa Variance
0.357143 0.002343

Obs Correct Total Obs % Observed Correct
10 18 55.555556

[1]
#validation input
590188.6875|4919453.8125||32
590188.6875|4918947||32
590307.9375|4918350.75||32
590576.25|4917694.875||32
590307.9375|4917694.875||32
590546.4375|4919364.375||32
590874.375|4919364.375||32
590904.1875|4919483.625||32
603604.3125|4922047.5||81
603336|4922315.8125||81
603514.875|4922464.875||81
603604.3125|4922256.1875||81
603634.125|4922107.125||81
603693.75|4921987.875||81
603962.0625|4921987.875||81
603753.375|4921600.3125||81
603962.0625|4922047.5||81
598983.375|4922256.1875||81

2

On Fri, Apr 1, 2011 at 2:45 PM, daniel mcinerney
<daniel.o.mcinerney@gmail.com> wrote:

Hi list,

I have a query regarding the error matrix produced
using r.kappa (in GRASS 6.4.0 RC6) and hoping that
the following will help someone to replicate my findings.
From what I see, the error matrix is mis-labeling the
rows/columns in the matrix.

Right, I noticed that some months ago, but fixed it only in grass 7.
Now also fixed in 6.5 and 6.4 (r45818 and r45819, respectively).

Markus M

Using the spearfish dataset, I digitised 18 points
to create a reference dataset to validate
the classification, in this case the raster 'landcover.orig'
For the purposes of the test and simplicity, I ensured
that the points were located within two landcover classes
(raster values 42 and 81). The following steps were carried out
to generate the output from r.kappa

#Create reference dataset
d.where | r.what input=landcover.orig > kappa_test.txt

For the purposes of this test, I deliberately changed the
landcover class from 42 to 32 in the validation dataset [1]
This will create an off-diagonal value in the error matrix.

#Create the vector point from ascii and convert to raster
v.in.ascii input=kappa_test output=kapp_test cat=4
v.to.rast input=kapp_test output=validation use=cat

#Run r.kappa
r.kappa classification=landcover.orig reference=validation
output=validation.txt

The output from r.kappa (validation.txt) is below, however, it appears
to me that the matrix is transposed, i.e. the horizontal values
(MAP1) should in fact be vertical. From my interpretation, the error
matrix indicates that 8 validation points with the value 42 were
classified as 32, but in fact it should state that 8 points with
the value of 32 were classified as 42? Can anyone confirm
my findings?

Thanks in advance.

Regards,
Daniel.

                    ACCURACY ASSESSMENT

LOCATION: spearfish60 Fri Apr 1 12:45:42 2011
MASK: none
MAPS: MAP1 = Categories (validation in user1)
MAP2 = (untitled) (landcover.orig in PERMANENT)

Error Matrix
Panel #1 of 1
MAP1
cat# 32 42 81 Row Sum
M 32 0 8 0 8
A 42 0 0 0 0
P 81 0 0 10 10
Col Sum 0 8 10 18

Cats % Commission % Ommission Estimated Kappa
32 100.000000 -nan -999.000000
42 NA NA NA
81 0.000000 0.000000 1.000000

Kappa Kappa Variance
0.357143 0.002343

Obs Correct Total Obs % Observed Correct
10 18 55.555556

[1]
#validation input
590188.6875|4919453.8125||32
590188.6875|4918947||32
590307.9375|4918350.75||32
590576.25|4917694.875||32
590307.9375|4917694.875||32
590546.4375|4919364.375||32
590874.375|4919364.375||32
590904.1875|4919483.625||32
603604.3125|4922047.5||81
603336|4922315.8125||81
603514.875|4922464.875||81
603604.3125|4922256.1875||81
603634.125|4922107.125||81
603693.75|4921987.875||81
603962.0625|4921987.875||81
603753.375|4921600.3125||81
603962.0625|4922047.5||81
598983.375|4922256.1875||81

_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

3

Thanks for your email Markus. That’s a necessary fix
to r.kappa - one that may not be immediately
obvious if the errors of omission and commission are more
evenly dispersed between classes.

Regarding the individual class errors, would it make sense to
simplify or standardise the values: -999.0000; -nan and NA
that are each reported within the r.kappa output?

Regards,
Daniel.

On Fri, Apr 1, 2011 at 3:42 PM, Markus Metz <markus.metz.giswork@googlemail.com> wrote:

On Fri, Apr 1, 2011 at 2:45 PM, daniel mcinerney
<daniel.o.mcinerney@gmail.com> wrote:

Hi list,

I have a query regarding the error matrix produced
using r.kappa (in GRASS 6.4.0 RC6) and hoping that
the following will help someone to replicate my findings.
From what I see, the error matrix is mis-labeling the
rows/columns in the matrix.

Right, I noticed that some months ago, but fixed it only in grass 7.
Now also fixed in 6.5 and 6.4 (r45818 and r45819, respectively).

Markus M

Using the spearfish dataset, I digitised 18 points
to create a reference dataset to validate
the classification, in this case the raster ‘landcover.orig’
For the purposes of the test and simplicity, I ensured
that the points were located within two landcover classes
(raster values 42 and 81). The following steps were carried out
to generate the output from r.kappa

#Create reference dataset
d.where | r.what input=landcover.orig > kappa_test.txt

For the purposes of this test, I deliberately changed the
landcover class from 42 to 32 in the validation dataset [1]
This will create an off-diagonal value in the error matrix.

#Create the vector point from ascii and convert to raster
v.in.ascii input=kappa_test output=kapp_test cat=4
v.to.rast input=kapp_test output=validation use=cat

#Run r.kappa
r.kappa classification=landcover.orig reference=validation
output=validation.txt

The output from r.kappa (validation.txt) is below, however, it appears
to me that the matrix is transposed, i.e. the horizontal values
(MAP1) should in fact be vertical. From my interpretation, the error
matrix indicates that 8 validation points with the value 42 were
classified as 32, but in fact it should state that 8 points with
the value of 32 were classified as 42? Can anyone confirm
my findings?

Thanks in advance.

Regards,
Daniel.

ACCURACY ASSESSMENT
LOCATION: spearfish60 Fri Apr 1 12:45:42 2011
MASK: none
MAPS: MAP1 = Categories (validation in user1)
MAP2 = (untitled) (landcover.orig in PERMANENT)

Error Matrix
Panel #1 of 1
MAP1
cat# 32 42 81 Row Sum
M 32 0 8 0 8
A 42 0 0 0 0
P 81 0 0 10 10
Col Sum 0 8 10 18

Cats % Commission % Ommission Estimated Kappa
32 100.000000 -nan -999.000000
42 NA NA NA
81 0.000000 0.000000 1.000000

Kappa Kappa Variance
0.357143 0.002343

Obs Correct Total Obs % Observed Correct
10 18 55.555556

[1]
#validation input
590188.6875|4919453.8125||32
590188.6875|4918947||32
590307.9375|4918350.75||32
590576.25|4917694.875||32
590307.9375|4917694.875||32
590546.4375|4919364.375||32
590874.375|4919364.375||32
590904.1875|4919483.625||32
603604.3125|4922047.5||81
603336|4922315.8125||81
603514.875|4922464.875||81
603604.3125|4922256.1875||81
603634.125|4922107.125||81
603693.75|4921987.875||81
603962.0625|4921987.875||81
603753.375|4921600.3125||81
603962.0625|4922047.5||81
598983.375|4922256.1875||81

grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user