[GRASS-dev] i.gensiget: "Unreliable clustering. Try a smaller initial number of clusters"

Nikos_Alexandris · June 28, 2013, 2:40pm

Hi!

Today using i.gensigset, I bombed into this strange behaviour as the one
discussed in the past

<http://lists.osgeo.org/pipermail/grass-user/2010-April/055639.html>

where the module did not generate any signatures. Anyone that knows what's
going on? I have tried various stuff without success.

I also tried in the "imagery60/user1" location/mapset using the Spot subsets
(spot.ms.1, spot.ms2, spot.ms.3), and it just works fine!

Thanks, N
---

A test-case:

# where am I
g.proj -p

-PROJ_INFO-------------------------------------------------
name : Universal Transverse Mercator
proj : utm
datum : wgs84
ellps : wgs84
zone : 48
no_defs : defined
-PROJ_UNITS------------------------------------------------
unit : metre
units : metres
meters : 1

# region
g.region -p

projection: 1 (UTM)
zone: 48
datum: wgs84
ellipsoid: wgs84
north: 178800
south: 137200
west: 163500
east: 225200
nsres: 100
ewres: 100
rows: 416
cols: 617
cells: 256672

# Top of Canopy Reflectance (from a Landsat 7 scene)
for Band in `g.mlist type=rast pat=ToCR?`; do r.info -r ${Band}; done

raster map(s) available in mapset <classification_trials>:
min=0.00124375210840126
max=0.35849866608563
min=0
max=0.35782345515399
min=0
max=0.379788238688478
min=0.00243176894430602
max=0.467877978869486
min=0.00498972477821269
max=0.405811742521198
min=263.377037572447
max=301.084746200043
min=0.00636355695948437
max=0.457282493983423

# a random samples map lies inside
r.info -gr random_trainingmap

north=178800
south=137200
east=225200
west=163500
nsres=100
ewres=100
rows=416
cols=617
cells=256672
datatype=CELL
min=1
max=5

# and is not empty
r.stats -cp random_trainingmap
100%

1 155 0.06%
2 203 0.08%
3 90 0.04%
4 45 0.02%
5 104 0.04%
* 256075 99.77%

# grouping ToCRs
i.group group=spectral subgroup=123457
input=ToCR1,ToCR2,ToCR3,ToCR4,ToCR5,ToCR7

[..]

# attempting to generate signatures
i.gensigset trainingmap=random_trainingmap group=spectral subgroup=123457
signaturefile=123457

Finding training classes...
100%
5 classes found
Reading raster maps...
100%
Clustering class 1 (155 pixels)...
WARNING: Removed a singular subsignature number 1 (4 remain)
WARNING: Removed a singular subsignature number 1 (3 remain)
WARNING: Removed a singular subsignature number 1 (2 remain)
WARNING: Removed a singular subsignature number 1 (1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
WARNING: Removed a singular subsignature number 1 (-1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
Number of subclasses is 0
Clustering class 2 (203 pixels)...
WARNING: Removed a singular subsignature number 1 (4 remain)
WARNING: Removed a singular subsignature number 1 (3 remain)
WARNING: Removed a singular subsignature number 1 (2 remain)
WARNING: Removed a singular subsignature number 1 (1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
WARNING: Removed a singular subsignature number 1 (-1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
Number of subclasses is 0
Clustering class 3 (90 pixels)...
WARNING: Removed a singular subsignature number 1 (4 remain)
WARNING: Removed a singular subsignature number 1 (3 remain)
WARNING: Removed a singular subsignature number 1 (2 remain)
WARNING: Removed a singular subsignature number 1 (1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
WARNING: Removed a singular subsignature number 1 (-1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
Number of subclasses is 0
Clustering class 4 (45 pixels)...
WARNING: Too many subclasses for class index 4
Number of subclasses set to 4
WARNING: Removed a singular subsignature number 1 (3 remain)
WARNING: Removed a singular subsignature number 1 (2 remain)
WARNING: Removed a singular subsignature number 1 (1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
WARNING: Removed a singular subsignature number 1 (-1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
Number of subclasses is 0
Clustering class 5 (104 pixels)...
WARNING: Removed a singular subsignature number 1 (4 remain)
WARNING: Removed a singular subsignature number 1 (3 remain)
WARNING: Removed a singular subsignature number 1 (2 remain)
WARNING: Removed a singular subsignature number 1 (1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
WARNING: Removed a singular subsignature number 1 (-1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
Number of subclasses is 0
i.gensigset complete.

Blumentrath_Stefan1 · June 28, 2013, 8:50pm

Hi Nikos

I had similar problems with this module. My experience with i.gensigset is, that it can be quite sensitive for how training areas aredistributed and how well they can be distinguished.
I do not know the algorithm nor what exactly the problem was in my case. But I solved this error message by adjusting the training areas in a way that I added training areas for classes where I had a relatively lower number of pixels (compared to the number of pixels in other classes) in a first try and removed/reshaped possibly conflicting training units in order to avoid overapp in subsignatures across classes.
I do not know how you generated your training areas, but maybe you have a chance to try adjusting them...

Hope that helps and I am interested in what does the trick for you or what other, more experienced, users say on that topic...

Cheers
Stefan
________________________________________
Von: grass-dev-bounces@lists.osgeo.org [grass-dev-bounces@lists.osgeo.org]" im Auftrag von "Nikos Alexandris [nik@nikosalexandris.net]
Gesendet: Freitag, 28. Juni 2013 16:40
An: GRASS developers list
Betreff: [GRASS-dev] i.gensiget: "Unreliable clustering. Try a smaller initial number of clusters"

Hi!

Today using i.gensigset, I bombed into this strange behaviour as the one
discussed in the past

<http://lists.osgeo.org/pipermail/grass-user/2010-April/055639.html>

where the module did not generate any signatures. Anyone that knows what's
going on? I have tried various stuff without success.

I also tried in the "imagery60/user1" location/mapset using the Spot subsets
(spot.ms.1, spot.ms2, spot.ms.3), and it just works fine!

Thanks, N
---

A test-case:

# where am I
g.proj -p

-PROJ_INFO-------------------------------------------------
name : Universal Transverse Mercator
proj : utm
datum : wgs84
ellps : wgs84
zone : 48
no_defs : defined
-PROJ_UNITS------------------------------------------------
unit : metre
units : metres
meters : 1

# region
g.region -p

projection: 1 (UTM)
zone: 48
datum: wgs84
ellipsoid: wgs84
north: 178800
south: 137200
west: 163500
east: 225200
nsres: 100
ewres: 100
rows: 416
cols: 617
cells: 256672

# Top of Canopy Reflectance (from a Landsat 7 scene)
for Band in `g.mlist type=rast pat=ToCR?`; do r.info -r ${Band}; done

raster map(s) available in mapset <classification_trials>:
min=0.00124375210840126
max=0.35849866608563
min=0
max=0.35782345515399
min=0
max=0.379788238688478
min=0.00243176894430602
max=0.467877978869486
min=0.00498972477821269
max=0.405811742521198
min=263.377037572447
max=301.084746200043
min=0.00636355695948437
max=0.457282493983423

# a random samples map lies inside
r.info -gr random_trainingmap

north=178800
south=137200
east=225200
west=163500
nsres=100
ewres=100
rows=416
cols=617
cells=256672
datatype=CELL
min=1
max=5

# and is not empty
r.stats -cp random_trainingmap
100%

1 155 0.06%
2 203 0.08%
3 90 0.04%
4 45 0.02%
5 104 0.04%
* 256075 99.77%

# grouping ToCRs
i.group group=spectral subgroup=123457
input=ToCR1,ToCR2,ToCR3,ToCR4,ToCR5,ToCR7

[..]

# attempting to generate signatures
i.gensigset trainingmap=random_trainingmap group=spectral subgroup=123457
signaturefile=123457

Finding training classes...
100%
5 classes found
Reading raster maps...
100%
Clustering class 1 (155 pixels)...
WARNING: Removed a singular subsignature number 1 (4 remain)
WARNING: Removed a singular subsignature number 1 (3 remain)
WARNING: Removed a singular subsignature number 1 (2 remain)
WARNING: Removed a singular subsignature number 1 (1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
WARNING: Removed a singular subsignature number 1 (-1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
Number of subclasses is 0
Clustering class 2 (203 pixels)...
WARNING: Removed a singular subsignature number 1 (4 remain)
WARNING: Removed a singular subsignature number 1 (3 remain)
WARNING: Removed a singular subsignature number 1 (2 remain)
WARNING: Removed a singular subsignature number 1 (1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
WARNING: Removed a singular subsignature number 1 (-1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
Number of subclasses is 0
Clustering class 3 (90 pixels)...
WARNING: Removed a singular subsignature number 1 (4 remain)
WARNING: Removed a singular subsignature number 1 (3 remain)
WARNING: Removed a singular subsignature number 1 (2 remain)
WARNING: Removed a singular subsignature number 1 (1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
WARNING: Removed a singular subsignature number 1 (-1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
Number of subclasses is 0
Clustering class 4 (45 pixels)...
WARNING: Too many subclasses for class index 4
Number of subclasses set to 4
WARNING: Removed a singular subsignature number 1 (3 remain)
WARNING: Removed a singular subsignature number 1 (2 remain)
WARNING: Removed a singular subsignature number 1 (1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
WARNING: Removed a singular subsignature number 1 (-1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
Number of subclasses is 0
Clustering class 5 (104 pixels)...
WARNING: Removed a singular subsignature number 1 (4 remain)
WARNING: Removed a singular subsignature number 1 (3 remain)
WARNING: Removed a singular subsignature number 1 (2 remain)
WARNING: Removed a singular subsignature number 1 (1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
WARNING: Removed a singular subsignature number 1 (-1 remain)
WARNING: Unreliable clustering. Try a smaller initial number of clusters
Number of subclasses is 0
i.gensigset complete.
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Nikos_Alexandris · July 6, 2013, 12:41am

Blumentrath, Stefan wrote:

Hi Nikos

Hallo Stefan,

I had similar problems with this module. My experience with i.gensigset is,
that it can be quite sensitive for how training areas aredistributed and
how well they can be distinguished.

Interesting. The only "failure" I was able, so far, to verify is that the
training areas should not capture NULL pixels in the to-be-classified
image(s). Can't trace though my posts about this (few years back).

I do not know the algorithm nor what exactly the problem was in my case. But
I solved this error message by adjusting the training areas in a way that I
added training areas for classes where I had a relatively lower number of
pixels (compared to the number of pixels in other classes) in a first try
and removed/reshaped possibly conflicting training units in order to avoid
overapp in subsignatures across classes.

Will try (soon I think == within the next 2 weeks).

I do not know how you generated your training areas, but maybe you have a
chance to try adjusting them...

It was a (quick) test only, I remember: created 3 sets of random points (some
MASKed region, thus no NULL pixels included), buffered them to rational
radiuses, labeled them (one 'val' for each set), grouped them, fed
i.gensigset.

Hope that helps and I am interested in what does the trick for you or what
other, more experienced, users say on that topic...

Will feed-back when the time comes.

Danke, Nikos

mlennert · July 6, 2013, 11:54am

On Sat, July 6, 2013 02:41, Nikos Alexandris wrote:

It was a (quick) test only, I remember: created 3 sets of random points
(some
MASKed region, thus no NULL pixels included), buffered them to rational
radiuses, labeled them (one 'val' for each set), grouped them, fed
i.gensigset.

This might explain the issue ("unrelable clustering"): if your random
buffer areas are not real training areas representing specific homogenous
types of spectral responses, but completely random collections of spectral
signatures, how could any reliable generation of significanly
distinguished classes be achieved ?

You have to use training areas that actually have a meaning, i.e. be
representative of the different classes you wish to identify.

Moritz

Nikos_Alexandris · July 6, 2013, 12:05pm

Nikos Alexandris wrote:

> It was a (quick) test only, I remember: created 3 sets of random points
> (some
> MASKed region, thus no NULL pixels included), buffered them to rational
> radiuses, labeled them (one 'val' for each set), grouped them, fed
> i.gensigset.

Moritz Lennert wrote:

This might explain the issue ("unrelable clustering"): if your random
buffer areas are not real training areas representing specific homogenous
types of spectral responses, but completely random collections of spectral
signatures, how could any reliable generation of significanly
distinguished classes be achieved ?

You have to use training areas that actually have a meaning, i.e. be
representative of the different classes you wish to identify.

Yes, maybe. Note(d), I ran the same test-process in "spearfish60" where it
worked fine -- luck?

Nikos