[GRASS5] New thematic mapper for GRASS 6

I’ve just finished a thematic mapper for vector maps in GRASS 6, d.vect.thematic, and am hoping that some of you can help test it. It’s done as a shell script (actually 2 scripts). These can be dropped into the $GISBASE/scripts folder.

D.vect.thematic will create thematic vector maps of graduated colors or graduated point sizes by regular intervals, standard deviation units, or quartiles. It creates a legend of sorts in the output area of the GUI dialog, that can be cut and pasted into another application. Currently it only works for numeric fields in the connected attribute table, like d.vect.chart. However, I hope that this still will be helpful to GRASS users and respond to the criticism that GRASS lacks thematic mapping.

To do this, I had to create a script-based stats program, v.univar.sh, that is based on r.univar.sh. Markus has suggested that it would be much faster if I could use the C-code v.univar module. I’d be happy to, but it needs to be updated somewhat. I need to be able to get the mean and standard deviation for all vector types, not just points. I’m not sure why that limitation is there—maybe a legacy of s.univar. I also need to get quartiles. If someone can do this, I can modify d.vect.thematic to use v.univar instead of v.univar.sh.

The same thing goes for r.univar. It would be pretty easy to create a d.rast.thematic using an updated r.univar (with quartiles).

Also, maybe someone can come up with a way to make a nicer legend for d.vect.thematic. It would be great to have actual colors (as points, lines, and areas) or point icons showing rather than just the R:G:B code or point size (as I’ve done here). Maybe someone with a bit more tcltk skills could do this.

Let me know what you think. If it seems to work for everyone, I’ll go ahead and add it to the CVS and the GIS Manager menus.

Michael


C. Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
PO Box 872402
Arizona State University
Tempe, AZ 85287-2402
USA

Phone: 480-965-6262
Fax: 480-965-7671
www: <www.public.asu.edu/~cmbarton>

(attachments)

thematic.tgz (3.87 KB)

Michael,

On Wed, May 11, 2005 7:39, Michael Barton said:

I¹ve just finished a thematic mapper for vector maps in GRASS 6,

d.vect.thematic, and am hoping that some of you can help test it.

This is way cool !

I had been thinking about something like this for a while, but as always, I
think too much and do too little :wink:

First superficial testing shows it works great, with just a few remarks:

- One problem I encounter sometimes (still trying to figure out the exact
sequence leading to this) is the following error:

/usr/local/grass-6.1.cvs/scripts/d.vect.thematic: line 143: TMP1: Permission
denied

and then nothing is displayed as the file suppoed to contain the statistics is
empty.

- The options available in the key "type" should be reduced to the only
sensible ones (point, centroid, area) as the others don't seem to work (or I
misunderstood something); also it might be useful to print a warning if the
type of thematic map display is set to "graduated points" and the vector map
type to "area" and vice versa. Currently the program runs, but nothing shows
up.

- Concerning the legend: it would be good to have a precise indication of
where one category ends and the other starts. At the moment, it prints
something like

1.000000-3.2725 5
3.2725-5.545 7
5.545-7.8175 9
7.8175-10.09 11

so if you have a value of 3.2725, you don't know in which category it is...
If I understand the code correctly it is in the first category and the legend
should therefore be:

1.000000-3.2725 5
3.2726-5.545 7

or some other way of saying that the range goes from minimum (exclusive) to
maximum (inclusive).
BTW the way, this raises the question of what happens if a feature has exactly
the $min value of the first category ?

And just to prove once again that users are a terribly ungrateful bunch and
always want more, here a few wishes:

- I think one useful addition would be the possibility to display the
available columns, or, even better, to chose from a list of available columns.

- It was very easy for me to very quickly create a new color scheme (here we
use blue for low values and red for high values...) in the script. That's very
useful for those of us who are not afraid of going into the code. But it would
be interesting to see if there is a way to offer a color choser to chose the
min and max color and to automatically calculate the intermediate values.
Another great option would be to also be able to chose all the colors
manually...

Congrats Michael, this will make GRASS much usable for all the vector bunch
out there !

Moritz

Moritz,

Thanks for the info and testing. I'll quickly reply below.

Michael
______________________________
Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
Arizona State University
Tempe, AZ 85287-2402
USA

voice: 480-965-6262; fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: Moritz Lennert <mlennert@club.worldonline.be>
Reply-To: <mlennert@club.worldonline.be>
Date: Wed, 11 May 2005 15:08:43 +0200 (CEST)
To: Michael Barton <michael.barton@asu.edu>
Cc: grass5 developers list <grass5@grass.itc.it>
Subject: Re: [GRASS5] New thematic mapper for GRASS 6

Michael,

On Wed, May 11, 2005 7:39, Michael Barton said:

I¹ve just finished a thematic mapper for vector maps in GRASS 6,

d.vect.thematic, and am hoping that some of you can help test it.

This is way cool !

I had been thinking about something like this for a while, but as always, I
think too much and do too little :wink:

First superficial testing shows it works great, with just a few remarks:

- One problem I encounter sometimes (still trying to figure out the exact
sequence leading to this) is the following error:

/usr/local/grass-6.1.cvs/scripts/d.vect.thematic: line 143: TMP1: Permission
denied

I am using g.temp to generate a TMP file ID and create a TMP file to hold
data needed to calculate stats. This is actually taking place in v.univar.sh
and would be unnecessary if v.univar can be fixed/upgraded. I'm using the
format that is required for proper security in creating temporary files. I'm
guessing that there is a problem on your system in writing a file in the
appropriate locale. Is there possibly another TMP1 already created (by
another process) and in use in this area?

and then nothing is displayed as the file suppoed to contain the statistics is
empty.

See above

- The options available in the key "type" should be reduced to the only
sensible ones (point, centroid, area) as the others don't seem to work (or I
misunderstood something); also it might be useful to print a warning if the
type of thematic map display is set to "graduated points" and the vector map
type to "area" and vice versa. Currently the program runs, but nothing shows
up.

In fact, lines and boundaries should also display with graduated colors
here. So they do make sense. I haven't tested it with lines yet however.

- Concerning the legend: it would be good to have a precise indication of
where one category ends and the other starts. At the moment, it prints
something like

1.000000-3.2725 5
3.2725-5.545 7
5.545-7.8175 9
7.8175-10.09 11

The algorithm specifies the range (given above) then looks for records > 1st
number and <= 2nd number. I understand what you are wanting to do, but not
sure how to do it. I'll see if I can figure out something as I refine this.

so if you have a value of 3.2725, you don't know in which category it is...
If I understand the code correctly it is in the first category and the legend
should therefore be:

1.000000-3.2725 5
3.2726-5.545 7

or some other way of saying that the range goes from minimum (exclusive) to
maximum (inclusive).
BTW the way, this raises the question of what happens if a feature has exactly
the $min value of the first category ?

Right now it gets left out. But this I can change (I think).

And just to prove once again that users are a terribly ungrateful bunch and
always want more, here a few wishes:

- I think one useful addition would be the possibility to display the
available columns, or, even better, to chose from a list of available columns.

I'd love to do this, but don't know how to program this in the existing
g.parser interface via shell scripting at the moment.

- It was very easy for me to very quickly create a new color scheme (here we
use blue for low values and red for high values...) in the script. That's very
useful for those of us who are not afraid of going into the code. But it would
be interesting to see if there is a way to offer a color choser to chose the
min and max color and to automatically calculate the intermediate values.
Another great option would be to also be able to chose all the colors
manually...

I'd also like a color chooser (kind of like in r.colors rules). I have an
idea of how this might be accomplished. I don't know if this would make it
harder (i.e., too complicated) or easier for most people to do.

I thought about a manual color chooser, but realized that by making a group
in the GIS Manager and duplicating a vector display line, there is already a
much better interface for doing this than I could do in a script.

Congrats Michael, this will make GRASS much usable for all the vector bunch
out there !

Thanks for your input. It is indeed appreciated. I'm glad if it helps those
who are using vectors.

Now if someone could just figure out how to re-write d.legend for vectors
also....

Michael

Moritz

Hi,

thanks a lot! - it is really useful tool...

I have two comments

  * it is working only for dbf driver (see patch)
        * maybe it would be good to distinguish between map name and table name
    - so to add a new parameter - table

Martin

--
Martin Landa <landa@gama.fsv.cvut.cz> * http://gama.fsv.cvut.cz/~landa *
CTU Prague, Faculty of Civil Engineering, Czech Republic

(attachments)

d.vect.thematic.patch (326 Bytes)

On Wed, May 11, 2005 21:14, Michael Barton said:

From: Moritz Lennert <mlennert@club.worldonline.be>

- One problem I encounter sometimes (still trying to figure out the exact
sequence leading to this) is the following error:

/usr/local/grass-6.1.cvs/scripts/d.vect.thematic: line 143: TMP1: Permission
denied

I am using g.temp to generate a TMP file ID and create a TMP file to hold
data needed to calculate stats. This is actually taking place in v.univar.sh
and would be unnecessary if v.univar can be fixed/upgraded. I'm using the
format that is required for proper security in creating temporary files. I'm
guessing that there is a problem on your system in writing a file in the
appropriate locale. Is there possibly another TMP1 already created (by
another process) and in use in this area?

Must be something like this, but I haven't been able to pin it down, yet. And
unfortunately I won't have much time for testing in the next few days...

- The options available in the key "type" should be reduced to the only
sensible ones (point, centroid, area) as the others don't seem to work (or I
misunderstood something); also it might be useful to print a warning if the
type of thematic map display is set to "graduated points" and the vector map
type to "area" and vice versa. Currently the program runs, but nothing shows
up.

In fact, lines and boundaries should also display with graduated colors
here. So they do make sense. I haven't tested it with lines yet however.

Sorry, my error. yes it seems to work well with lines. Haven't had the time to
try with boundaries...

- Concerning the legend: it would be good to have a precise indication of
where one category ends and the other starts. At the moment, it prints
something like

1.000000-3.2725 5
3.2725-5.545 7
5.545-7.8175 9
7.8175-10.09 11

The algorithm specifies the range (given above) then looks for records > 1st
number and <= 2nd number. I understand what you are wanting to do, but not
sure how to do it. I'll see if I can figure out something as I refine this.

I think it is mostly a question of display. Something like this might already
be alright:

[1.000000-3.2725] 5
]3.2725-5.545] 7
]5.545-7.8175] 9

And just to prove once again that users are a terribly ungrateful bunch and
always want more, here a few wishes:

- I think one useful addition would be the possibility to display the
available columns, or, even better, to chose from a list of available
columns.

I'd love to do this, but don't know how to program this in the existing
g.parser interface via shell scripting at the moment.

Sorry, can't help you there.

- It was very easy for me to very quickly create a new color scheme (here we
use blue for low values and red for high values...) in the script. That's
very
useful for those of us who are not afraid of going into the code. But it
would
be interesting to see if there is a way to offer a color choser to chose the
min and max color and to automatically calculate the intermediate values.
Another great option would be to also be able to chose all the colors
manually...

I'd also like a color chooser (kind of like in r.colors rules). I have an
idea of how this might be accomplished. I don't know if this would make it
harder (i.e., too complicated) or easier for most people to do.

I thought about a manual color chooser, but realized that by making a group
in the GIS Manager and duplicating a vector display line, there is already a
much better interface for doing this than I could do in a script.

I'm not sure I understand exactly what you mean by this, but it seems more
complicated, as you have to define the where clause in each line, whereas
d.vect.thematic calculates the intervals automatically.

Another cool feature would be the possibility to enter interval boundaries
manually...

Moritz

Also, given v.univar.sh, the quartile method won't work properly with
odd-sized data. Change line 149 from

q2="`grep 'Median' TMP1 | sed s/'Median (even N):'//`"

to

q2="`grep 'Median' TMP1 | sed s/'Median (.* N):'//`"

I'd say for the "custom" colormaps, you might set an auxiliary file for
definitions, something like:

red-blue (tab) 255:0:0 (tab) 0:0:255
orange-white (tab) 255:128:0 (tab) 255:255:255

and read that from the script. Awk will do that with no problem. I don't know
where you would put such a file, though... $GRASS/etc, maybe?

Thanks a lot for this one.

Daniel.

-- Daniel Calvelo Aros

---------- Original Message -----------
From: Martin Landa <landa@gama.fsv.cvut.cz>
To: GRASS 5 Developers mailing list <grass5@grass.itc.it>
Cc: Michael Barton <michael.barton@asu.edu>
Sent: Wed, 11 May 2005 22:50:33 +0200
Subject: Re: [GRASS5] New thematic mapper for GRASS 6

Hi,

thanks a lot! - it is really useful tool...

I have two comments

  * it is working only for dbf driver (see patch)
        * maybe it would be good to distinguish between map name and
table name - so to add a new parameter - table

Martin

--
Martin Landa <landa@gama.fsv.cvut.cz> *
http://gama.fsv.cvut.cz/~landa * CTU Prague, Faculty of Civil
Engineering, Czech Republic

------- End of Original Message -------

Hi,

maybe I am wrong, but wouldn't be possible to determine table name from the layer option?
Is the table option really needed?

Martin

--
Martin Landa <landa@gama.fsv.cvut.cz> * http://gama.fsv.cvut.cz/~landa *
CTU Prague, Faculty of Civil Engineering, Czech Republic

(attachments)

d.vect.thematic.patch (1.46 KB)