[GRASS-dev] porting r.in.wms to python

[cmbarton]

A few comments below.

On Oct 6, 2008, at 1:49 AM, <grass-dev-request@lists.osgeo.org> <grass-dev-request@lists.osgeo.org> wrote:

Date: Mon, 6 Oct 2008 06:34:41 +0100
From: Glynn Clements <glynn@gclements.plus.com>
Subject: Re: [GRASS-dev] porting r.in.wms to python
To: hamish_b@yahoo.com
Cc: grass-dev <grass-dev@lists.osgeo.org>
Message-ID: <18665.41841.122863.646844@cerise.gclements.plus.com>
Content-Type: text/plain; charset=us-ascii

Hamish wrote:

I was planning on leaving those until last

Well, I’m now planning on just leaving them, period

Here’s the current status on conversion of scripts to
Python:

The following scripts are disabled, and haven’t been converted:

d.out.gpsdrive (d.mon)

(me)
like the very useful d.out.file module, it dumps current (composed) map
display to the PNG driver via d.save. I suppose replacing this will
be a clone/modification of how d.out.file’s functionality has been
replicated.

IOW, it needs to be built into the GUI; now that monitors no longer
exist, nothing else has any notion of a “current” map.

It already exists in the GUI. It exists in the TclTk GUI too. If additional output formats are needed they can be added.

d.rast.leg (d.frame)

(any thoughts Markus?)

perhaps replaced by some wxGUI “load cartographic template” tool?
(users could contribute their own etc…)

Note that it is possible to set a drawing frame by setting
GRASS_FRAME=t,b,l,r (see d.polar for an example).

Actually, this should be salvageable without too much effort. I just
saw the d.frame calls and didn’t look much further.

Could this be wrapped into ps.map? If I remember, this simply puts a raster on the screen in one frame, a title in another, and a legend in a third. Doing this in the wxPython canvas is doable, but I wonder if a script is the best way to go with it. That is, building it into a wxPython class in the display module seems to make more sense. An alternative is to have a script combine a raster, title, and legend in an output graphic file (png, pdf, or something). This could be done more easily.

i.spectral (d.mon, d.where)

It would be very easy to add a coord=x,y[,x2,y2,…] option to feed
r.what instead of using d.where. The d.mon check is just to ensure
that d.where will work.

That sounds simple enough.

More work would be to replace gnuplot with d.graph or whatever, like
is done for d.polar.

Is d.linegraph suitable?

There is a simple graphing module that comes with wxPython that is much more sophisticated than d.linegraph–which still assumes an xterm (although it can output to a graphic file I suppose). However, I suggested that matplotlib might be quite suitable for this kind of task. It is a free, pure Python library that can do very sophisticated graphing very easily. It can be 1) wrapped into the GUI display, 2) set to create it’s own simple display in a couple of GUI toolkits, or 2) set to output to a graphic file. It is scriptable. I submitted a couple of test scripts to show how this might be accomplished.

These:
r.in.wms/r.in.wms
r.tileset/r.tileset

I think it is ok for the replacement version to depend on GDAL > 1.5.0.
A first step will be adding XML+GDAL option to the bash/sh version
for testing of that method.

It might be simpler to just start from scratch in Python.

v.in.gpsbabel/v.in.gpsbabel
v.in.garmin/v.in.garmin

Certainly v.in.gpsbabel should survive in some form. v.in.garmin is
useful as the gpsbabel garmin filter does not pass through all fields.

the XML stuff in v.in.gpsbabel is in desperate need of a python
replacement, otherwise it shouldn’t be /too/ different from v.in.mapgen.

Right. v.in.mapgen was another one that I was going to leave, but I
ended up downloading some sample files. That made it much easier to
see what the code was trying to do.

are too complex to replace using “rote” conversion (i.e. simply
replacing each chunk of code with equivalent Python code).

The above scripts really need to be rewritten by someone
who understands the overall purpose of the script.

For v.in.gps I guess that means me, but my python+xml is rather weak.

The Python documentation has an example at:

http://docs.python.org/library/xml.dom.minidom.html

Or I can write the code to parse the XML if you can explain the
overall structure. It’s just that trying to deduce the data structure
from a sequence of grep/head/tail/cut commands is rather mind-numbing.

For a Python version r.in.wms, I defer the larger project to someone
else, but can try to add GDAL support to the existing sh/bash version
to demonstrate/verify the method.

That probably isn’t much help. It would probably be more useful to
simply describe the process and provide some example URLs. I find
English much easier to read than bash/grep/head/tail/cut/awk/sed.

[…]
etc/gui/scripts/d.colors.sh
etc/gui/scripts/d.path.sh
etc/gui/scripts/r.colors.rules
etc/gui/scripts/r.reclass.file
etc/gui/scripts/r.reclass.rules
etc/gui/scripts/r.recode.file
etc/gui/scripts/r.recode.rules

these are all there as wrapper code between the GUI and command line
modules, usually WRT piping info from stdin. All should be replaced by
integrated wxGUI wizards not simply python scripts AFAICT.

Right. The various *.rules scripts plus d.colors.sh invoke xterm,
while the *.file versions just redirect stdin from a file (to
compensate for the lack of a file= option).

These are unneeded I think.

So, r.reclass and r.recode each need a file= (G_OPT_F_INPUT) option.

I thought this was already done.

d.path.sh combines d.vect and d.path. The d.vect step was necessary
when d.path used the mouse to obtain the coordinates, so that the user
could see what they were supposed to be clicking on. That’s no longer
applicable.

etc/gui/scripts/r.support.sh was partially needed because

1. non-interactive version did not in the past support all interactive
   functionality, and
2. interactive version weirdly exits prematurely when called directly
   from Tcl/Tk.

More generally, is there any intention to fix up gis.m with regard to
the various changes in 7.0, or are we going to abandon it and rely
upon wxgui instead?

The TclTk GUI is set to be abandoned in GRASS 7. It will continue to live in the GRASS 6 series.

Michael

[Glynn_Clements1]

Michael Barton wrote:

> > > d.rast.leg (d.frame)
> >
> > (any thoughts Markus?)
> >
> > perhaps replaced by some wxGUI "load cartographic template" tool?
> > (users could contribute their own etc..)
>
> Note that it is possible to set a drawing frame by setting
> GRASS_FRAME=t,b,l,r (see d.polar for an example).
>
> Actually, this should be salvageable without too much effort. I just
> saw the d.frame calls and didn't look much further.

Could this be wrapped into ps.map?

Well, I'm planning on making ps.map obsolete. It's just a matter of
determining what functionality is only available via ps.map and adding
or extending d.* commands to provide that functionality.

If I remember, this simply puts a
raster on the screen in one frame, a title in another, and a legend in
a third. Doing this in the wxPython canvas is doable, but I wonder if
a script is the best way to go with it. That is, building it into a
wxPython class in the display module seems to make more sense. An
alternative is to have a script combine a raster, title, and legend in
an output graphic file (png, pdf, or something). This could be done
more easily.

FWIW, I've converted d.rast.leg to Python as described above.

> > > i.spectral (d.mon, d.where)
> >
> > It would be very easy to add a coord=x,y[,x2,y2,...] option to feed
> > r.what instead of using d.where. The d.mon check is just to ensure
> > that d.where will work.
>
> That sounds simple enough.
>
> > More work would be to replace gnuplot with d.graph or whatever, like
> > is done for d.polar.
>
> Is d.linegraph suitable?

There is a simple graphing module that comes with wxPython that is
much more sophisticated than d.linegraph--which still assumes an xterm
(although it can output to a graphic file I suppose).

d.linegraph doesn't require an xmon.

AFAICT, d.linegraph can be used for i.spectral.

However, I
suggested that matplotlib might be quite suitable for this kind of
task. It is a free, pure Python library that can do very sophisticated
graphing very easily. It can be 1) wrapped into the GUI display, 2)
set to create it's own simple display in a couple of GUI toolkits, or
2) set to output to a graphic file. It is scriptable. I submitted a
couple of test scripts to show how this might be accomplished.

We would need to determine how to make it respect the various
environment variables so that commands which use matplotlib can be
mixed with those which use the display/raster libraries.

So far as "graphs" are concerned, I'd rather have a completely new
class of modules/scripts (e.g. st.*) which output structured data
rather than graphics. That way, the user would be able control the
display (scaling, tick intervals, log/linear scale, sybmology, ...)
without requiring each module to have a hundred options (or worse,
each module having some entirely arbitrary subset of the available
options).

> So, r.reclass and r.recode each need a file= (G_OPT_F_INPUT) option.

I thought this was already done.

It is; I've removed the wrapper scripts.

--
Glynn Clements <glynn@gclements.plus.com>

[Hamish]

Michael:

However, I suggested that matplotlib might be quite suitable for
this kind of task. It is a free, pure Python library that can do
very sophisticated graphing very easily.

a plus for matplotlib is that those of us who know Matlab plotting commands
quite well will already be trained in using it (to some extent). And others
can gain transferable knowledge.

Hamish

[cmbarton]

Thanks for the information Glynn. See below for a question or two.
On Oct 7, 2008, at 4:24 PM, Glynn Clements wrote:

Michael Barton wrote:

   d.rast.leg (d.frame)

(any thoughts Markus?)

perhaps replaced by some wxGUI "load cartographic template" tool?
(users could contribute their own etc..)

Note that it is possible to set a drawing frame by setting
GRASS_FRAME=t,b,l,r (see d.polar for an example).

Actually, this should be salvageable without too much effort. I just
saw the d.frame calls and didn't look much further.

Could this be wrapped into ps.map?

Well, I'm planning on making ps.map obsolete. It's just a matter of
determining what functionality is only available via ps.map and adding
or extending d.* commands to provide that functionality.

I'd guess the main things not included in d.* commands would be layout information, such as how to position more than one map on a page, how to position a scale or legend--especially if it is supposed to be located off the map display area--, and other similar things.

If I remember, this simply puts a
raster on the screen in one frame, a title in another, and a legend in
a third. Doing this in the wxPython canvas is doable, but I wonder if
a script is the best way to go with it. That is, building it into a
wxPython class in the display module seems to make more sense. An
alternative is to have a script combine a raster, title, and legend in
an output graphic file (png, pdf, or something). This could be done
more easily.

FWIW, I've converted d.rast.leg to Python as described above.

A nice shortcut to raster output.

   i.spectral (d.mon, d.where)

It would be very easy to add a coord=x,y[,x2,y2,...] option to feed
r.what instead of using d.where. The d.mon check is just to ensure
that d.where will work.

That sounds simple enough.

More work would be to replace gnuplot with d.graph or whatever, like
is done for d.polar.

Is d.linegraph suitable?

There is a simple graphing module that comes with wxPython that is
much more sophisticated than d.linegraph--which still assumes an xterm
(although it can output to a graphic file I suppose).

d.linegraph doesn't require an xmon.

It seems originally designed with an xmon in mind, but with your changes to display architecture, this is no longer important I guess.

AFAICT, d.linegraph can be used for i.spectral.

However, I
suggested that matplotlib might be quite suitable for this kind of
task. It is a free, pure Python library that can do very sophisticated
graphing very easily. It can be 1) wrapped into the GUI display, 2)
set to create it's own simple display in a couple of GUI toolkits, or
2) set to output to a graphic file. It is scriptable. I submitted a
couple of test scripts to show how this might be accomplished.

We would need to determine how to make it respect the various
environment variables so that commands which use matplotlib can be
mixed with those which use the display/raster libraries.

Sorry, but I don't understand. It is an external library (like gnuplot, but seems a bit easier to work with for me at least). It can plot to graphic files or a display canvas it creates, OR it can be included in the GUI (though this takes somewhat different programming to make it plot to the GUI display canvas). It was fairly easy to include this in scrips. But maybe you are thinking of using it in a more sophisticated way than I am.

So far as "graphs" are concerned, I'd rather have a completely new
class of modules/scripts (e.g. st.*) which output structured data
rather than graphics. That way, the user would be able control the
display (scaling, tick intervals, log/linear scale, sybmology, ...)
without requiring each module to have a hundred options (or worse,
each module having some entirely arbitrary subset of the available
options).

This would be excellent. I don't have a feel for how much programming effort this would take. If the development and maintenance effort can be supported, it would be very good to have a GRASS native plotting environment. This focuses on structured graphic output that is directly targeted for GIS needs--without the additional baggage of graph displays that would rarely or never be used.

So, r.reclass and r.recode each need a file= (G_OPT_F_INPUT) option.

I thought this was already done.

It is; I've removed the wrapper scripts.

Great. Thanks.

Michael

[Glynn_Clements1]

Michael Barton wrote:

>> However, I
>> suggested that matplotlib might be quite suitable for this kind of
>> task. It is a free, pure Python library that can do very
>> sophisticated
>> graphing very easily. It can be 1) wrapped into the GUI display, 2)
>> set to create it's own simple display in a couple of GUI toolkits, or
>> 2) set to output to a graphic file. It is scriptable. I submitted a
>> couple of test scripts to show how this might be accomplished.
>
> We would need to determine how to make it respect the various
> environment variables so that commands which use matplotlib can be
> mixed with those which use the display/raster libraries.

Sorry, but I don't understand. It is an external library (like
gnuplot, but seems a bit easier to work with for me at least). It can
plot to graphic files or a display canvas it creates, OR it can be
included in the GUI (though this takes somewhat different programming
to make it plot to the GUI display canvas). It was fairly easy to
include this in scrips. But maybe you are thinking of using it in a
more sophisticated way than I am.

The intention is to have a single graphics architecture whose
components can be combined in arbitrary ways. Not a bunch of disparate
scripts, generating different formats, recognising different options,
using different sets of fonts, etc.

AFAICT, we would need either a wrapper around matplotlib so that it
honours all of the GRASS_* variables related to graphics, or a
matplotlib "backend" which uses the display library.

--
Glynn Clements <glynn@gclements.plus.com>