[GRASS-dev] Shell scripts

GRASS GRASS developer
1

On Nov 25, 2009, at 10:27 PM, grass-dev-request@lists.osgeo.org wrote:

Date: Thu, 26 Nov 2009 01:25:48 +0000
From: Glynn Clements <glynn@gclements.plus.com>
Subject: Re: [GRASS-dev] Shell scripts
To: Markus Neteler <neteler@osgeo.org>
Cc: grass-dev@lists.osgeo.org
Message-ID: <19213.55580.942903.873211@cerise.gclements.plus.com>
Content-Type: text/plain; charset=us-ascii

Markus Neteler wrote:

The remaining question is whether d.path.sh and the p.* scripts should
be converted to Python or removed. I don't think that d.path.sh works
with the GUI. The p.* scripts appear to be an attempt to implement
d.mon-like behaviour for the GUI, but I don't know whether this is
functional.

The p.* scripts are semi-functional and should be rewritten to Python.
Not having d.mon/d.rast/d.vect/d.zoom commands is for me, Helena, others
the obstacle to not use GRASS 7 as primary version.
Since the needed (wx)GUI pieces are there, I still hope that someone skilled
is rewriting it in Python. For us it is essential to continue with command line
in GRASS 7.

d.rast and d.vect haven't gone anywhere.

If you want to be able to control the GUI from the command line, that
should be dealt with as an infrastructure issue, not by creating
wrappers around individual commands.

I can deal with the display/driver libraries, and with generic Python
IPC, but some of it will need the involvement of someone who
understands the GUI.

Winter break is near and I'll be laid up for a week. So I might be able to help.

Controlling the GUI from the command line is a contradiction in interface terms. Perhaps you really mean displaying a map from the command line?

If you don't want to use a mouse, there isn't much point in trying to work with the wxGUI map display canvas as it is--with all of its buttons and menus and a lot of other functionality built in. It's not impossible to break into it, but it is not easy either--which is why the p.* scripts remain non- or only semi-functional. Also, I suspect that command-line only display will be important primarily for users of Linux. So it seems like a better solution is to have a reasonably easy command line way to composite maps and display them (perhaps in pnm or png format) in a Linux-based viewer.

Glynn has mentioned the possibility of this and most of the tools already exist. Probably a python script could be created to 1) allow a user to specify a list of vector and raster maps to overlay, 2) set the output file grass variable and recursively cycle through d.vect and d.rast, 3) run the pnm files through g.pnmcomp to composite them, and 4) display the composited map in a Linux viewer utility of some sort. d.out.file might be useful as a model to get this started. If someone wants to attempt this, I'm happy to offer advice on the Python coding.

Another option could be a simplified interface to ps.map, enter maps only or maps and a couple of overlay options like a scale and north arrow, and use ps.map to do the compositing. Then display the result in a ps viewer. These could also be combined in a Python script. There used to be a TclTk script for most of this. This might produce higher quality visualization and has the added benefit of simultaneously producing high quality print files.

Michael

--
Glynn Clements <glynn@gclements.plus.com>

2

Michael Barton wrote:

> If you want to be able to control the GUI from the command line, that
> should be dealt with as an infrastructure issue, not by creating
> wrappers around individual commands.
>
> I can deal with the display/driver libraries, and with generic Python
> IPC, but some of it will need the involvement of someone who
> understands the GUI.

Winter break is near and I'll be laid up for a week. So I might be
able to help.

Controlling the GUI from the command line is a contradiction in
interface terms. Perhaps you really mean displaying a map from the
command line?

I mean controlling the GUI, i.e. being able to modify the list of
displayed maps. That's what the p.* scripts appear to be doing.

If you don't want to use a mouse, there isn't much point in trying to
work with the wxGUI map display canvas as it is--with all of its
buttons and menus and a lot of other functionality built in. It's not
impossible to break into it, but it is not easy either--which is why
the p.* scripts remain non- or only semi-functional. Also, I suspect
that command-line only display will be important primarily for users
of Linux. So it seems like a better solution is to have a reasonably
easy command line way to composite maps and display them (perhaps in
pnm or png format) in a Linux-based viewer.

Glynn has mentioned the possibility of this and most of the tools
already exist. Probably a python script could be created to 1) allow a
user to specify a list of vector and raster maps to overlay, 2) set
the output file grass variable and recursively cycle through d.vect
and d.rast, 3) run the pnm files through g.pnmcomp to composite them,
and 4) display the composited map in a Linux viewer utility of some
sort. d.out.file might be useful as a model to get this started. If
someone wants to attempt this, I'm happy to offer advice on the Python
coding.

Another option could be a simplified interface to ps.map, enter maps
only or maps and a couple of overlay options like a scale and north
arrow, and use ps.map to do the compositing. Then display the result
in a ps viewer. These could also be combined in a Python script. There
used to be a TclTk script for most of this. This might produce higher
quality visualization and has the added benefit of simultaneously
producing high quality print files.

Except, what appears to be desired is the ability to control wxGUI
from the command line, not a separate program for command-line use.

At the simplest level, the GUI just needs to implement the equivalent
of Tcl/Tk's "send", leaving the burden of constructing valid commands
with the client.

--
Glynn Clements <glynn@gclements.plus.com>

3

On Fri, Nov 27, 2009 at 2:56 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Michael Barton wrote:

> If you want to be able to control the GUI from the command line, that
> should be dealt with as an infrastructure issue, not by creating
> wrappers around individual commands.
>
> I can deal with the display/driver libraries, and with generic Python
> IPC, but some of it will need the involvement of someone who
> understands the GUI.

Winter break is near and I'll be laid up for a week. So I might be
able to help.

Controlling the GUI from the command line is a contradiction in
interface terms. Perhaps you really mean displaying a map from the
command line?

I mean controlling the GUI, i.e. being able to modify the list of
displayed maps. That's what the p.* scripts appear to be doing.

Yes, like what the d.* commands did with x0 etc.
I regularly enjoy the beauty of shell history which I would
not have using the (pure) GUI since clicks aren't registered.

Markus

4

If you take a look at histogram.py, this gives some idea of what it takes to create a module that renders in wxPython canvas. You can eliminate the code for drawing the histogram window and tool bar, of course, but there is still quite a bit of coding to do to run d.hist and have the results rendered in the canvas.

Something along this line could potentially be written to make it possible to display something in the canvas by typing a command of some sort from the terminal. But I was wondering if it would be simpler to display maps in another way for those who want to work in this way.

Also, did you know that you can type a d.vect or d.rast in the wxPython command console to add a map layer? Maybe a better way to go is to improve the wxPython command console with history, etc. than to create a separate interface.

Michael

On Nov 27, 2009, at 12:33 AM, Markus Neteler wrote:

On Fri, Nov 27, 2009 at 2:56 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Michael Barton wrote:

If you want to be able to control the GUI from the command line, that
should be dealt with as an infrastructure issue, not by creating
wrappers around individual commands.

I can deal with the display/driver libraries, and with generic Python
IPC, but some of it will need the involvement of someone who
understands the GUI.

Winter break is near and I'll be laid up for a week. So I might be
able to help.

Controlling the GUI from the command line is a contradiction in
interface terms. Perhaps you really mean displaying a map from the
command line?

I mean controlling the GUI, i.e. being able to modify the list of
displayed maps. That's what the p.* scripts appear to be doing.

Yes, like what the d.* commands did with x0 etc.
I regularly enjoy the beauty of shell history which I would
not have using the (pure) GUI since clicks aren't registered.

Markus

5

On Monday 30 November 2009, Michael Barton wrote:

If you take a look at histogram.py, this gives some idea of what it
takes to create a module that renders in wxPython canvas. You can
eliminate the code for drawing the histogram window and tool bar, of
course, but there is still quite a bit of coding to do to run d.hist
and have the results rendered in the canvas.

Something along this line could potentially be written to make it
possible to display something in the canvas by typing a command of
some sort from the terminal. But I was wondering if it would be
simpler to display maps in another way for those who want to work in
this way.

I think that this would be a great idea. I tend to share Markus' concerns
about preservation of the d.* commands. I agree that these are based on an
antiquated model of displaying information, but the concept is quite elegant
and in some ways superior to a GUI-based system. Perhaps there is a way to
leverage the canvas from the command line, without having to 1) re-invent the
wheel, and 2) duplicate efforts in the GUI. I understand that my ratio of
complaining : coding is not ideal-- so I would be willing to help test any
such mechanisms as they progress.

Also, did you know that you can type a d.vect or d.rast in the
wxPython command console to add a map layer? Maybe a better way to go
is to improve the wxPython command console with history, etc. than to
create a separate interface.

OK. Well, this is getting closer to what the purists among us would like, but
having this functionality in the console-- not the wx-based console-- is
important.

Cheers,
Dylan

Michael

On Nov 27, 2009, at 12:33 AM, Markus Neteler wrote:
> On Fri, Nov 27, 2009 at 2:56 AM, Glynn Clements
>
> <glynn@gclements.plus.com> wrote:
>> Michael Barton wrote:
>>>> If you want to be able to control the GUI from the command line,
>>>> that
>>>> should be dealt with as an infrastructure issue, not by creating
>>>> wrappers around individual commands.
>>>>
>>>> I can deal with the display/driver libraries, and with generic
>>>> Python
>>>> IPC, but some of it will need the involvement of someone who
>>>> understands the GUI.
>>>
>>> Winter break is near and I'll be laid up for a week. So I might be
>>> able to help.
>>>
>>> Controlling the GUI from the command line is a contradiction in
>>> interface terms. Perhaps you really mean displaying a map from the
>>> command line?
>>
>> I mean controlling the GUI, i.e. being able to modify the list of
>> displayed maps. That's what the p.* scripts appear to be doing.
>
> Yes, like what the d.* commands did with x0 etc.
> I regularly enjoy the beauty of shell history which I would
> not have using the (pure) GUI since clicks aren't registered.
>
> Markus

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

--
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341

6

On Nov 30, 2009, at 1:38 PM, Dylan Beaudette wrote:

On Monday 30 November 2009, Michael Barton wrote:

If you take a look at histogram.py, this gives some idea of what it
takes to create a module that renders in wxPython canvas. You can
eliminate the code for drawing the histogram window and tool bar, of
course, but there is still quite a bit of coding to do to run d.hist
and have the results rendered in the canvas.

Something along this line could potentially be written to make it
possible to display something in the canvas by typing a command of
some sort from the terminal. But I was wondering if it would be
simpler to display maps in another way for those who want to work in
this way.

I think that this would be a great idea. I tend to share Markus' concerns
about preservation of the d.* commands. I agree that these are based on an
antiquated model of displaying information, but the concept is quite elegant
and in some ways superior to a GUI-based system. Perhaps there is a way to
leverage the canvas from the command line, without having to 1) re-invent the
wheel, and 2) duplicate efforts in the GUI. I understand that my ratio of
complaining : coding is not ideal-- so I would be willing to help test any
such mechanisms as they progress.

Also, did you know that you can type a d.vect or d.rast in the
wxPython command console to add a map layer? Maybe a better way to go
is to improve the wxPython command console with history, etc. than to
create a separate interface.

OK. Well, this is getting closer to what the purists among us would like, but
having this functionality in the console-- not the wx-based console-- is
important.

Dylan,

I guess the point I was trying to make was that if some of the Linux folks don't want to use the GUI at all, but prefer to control GRASS entirely by typing commands in the system terminal, I think that it is easier to design a display mechanism from scratch and not bother trying to use the wxGUI canvas. That is, I'm not convinced that trying to "leverage" the wxGUI canvas won't be considerably more difficult than an alternative method from the system terminal. There is a lot of complex code in the GUI canvas and related wxPython modules to do composited rendering in an interactive environment that is not needed for simply displaying maps by typing commands. A solution focused on the system terminal is indeed a 'purist' one for Linux users, but probably won't be of much interest to most users outside of the Linux world and maybe not even to all Linux users.

Right now, you can set the following environment variables and run any d.* command to have an output file created of the map to be displayed.

GRASS_PNG_AUTO_WRITE = TRUE
GRASS_PNG_READ =FALSE
GRASS_TRUECOLOR = TRUE
GRASS_RENDER_IMMEDIATE = TRUE
GRASS_WIDTH = [value]
GRASS_HEIGHT = [value]

GRASS_CAIROFILE = [outputfile]
GRASS_PNGFILE = [outputfile]

For example, we have added functionality to NASA WorldWind that will display PNG files created in this way. Other viewing utilities could also be used even more easily, and the whole thing wrapped into a python script.

On the other hand, if these folks DO want to use the wxPython GUI (and all the interactive tools on the canvas) but simply want to be able to get maps displayed in the canvas by typing commands instead of adding them to the layer manager, then I think the better way to go is to enhance the wxPython console. This solution could be of interest to power users on all platforms.

Michael

Cheers,
Dylan

Michael

On Nov 27, 2009, at 12:33 AM, Markus Neteler wrote:

On Fri, Nov 27, 2009 at 2:56 AM, Glynn Clements

<glynn@gclements.plus.com> wrote:

Michael Barton wrote:

If you want to be able to control the GUI from the command line,
that
should be dealt with as an infrastructure issue, not by creating
wrappers around individual commands.

I can deal with the display/driver libraries, and with generic
Python
IPC, but some of it will need the involvement of someone who
understands the GUI.

Winter break is near and I'll be laid up for a week. So I might be
able to help.

Controlling the GUI from the command line is a contradiction in
interface terms. Perhaps you really mean displaying a map from the
command line?

I mean controlling the GUI, i.e. being able to modify the list of
displayed maps. That's what the p.* scripts appear to be doing.

Yes, like what the d.* commands did with x0 etc.
I regularly enjoy the beauty of shell history which I would
not have using the (pure) GUI since clicks aren't registered.

Markus

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
grass-dev Info Page

--
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341

7

Michael Barton wrote:

Right now, you can set the following environment variables and run any
d.* command to have an output file created of the map to be displayed.

GRASS_PNG_AUTO_WRITE = TRUE
GRASS_PNG_READ =FALSE
GRASS_TRUECOLOR = TRUE
GRASS_RENDER_IMMEDIATE = TRUE
GRASS_WIDTH = [value]
GRASS_HEIGHT = [value]

GRASS_CAIROFILE = [outputfile]
GRASS_PNGFILE = [outputfile]

Also, if you want each image to be overlaid upon the previous one
(like with the monitors), you can set GRASS_PNG_READ=TRUE once the
output file has been created. This causes each d.* command to start
with the "screen" initialised from the contents of the output file
(which must exist, and have the correct dimensions).

The same approach could be used by the GUI to skip the g.pnmcomp step
(provided that all layers have 100% opacity).

If you have an image viewer which can handle 32-bpp BMP files, you can
use GRASS_PNGFILE=map.bmp and GRASS_PNG_MAPPED to cause the file to be
mmap()d as the frame-buffer, which saves having to read and write the
file.

A suitable (although rather primitive) viewer is available as
visualisation/ximgview. This will refresh its display from the file
either at regular intervals, or when it receives SIGUSR1. If the
environment variable GRASS_NOTIFY exists, its value will be executed
via system() whenever a d.* command completes, so you can use e.g.:

  export "GRASS_NOTIFY=kill -USR1 `pidof ximgview`"

to force the display to be redrawn after each command.

Possibly even faster would be to use the cairo driver to render into
an existing X Pixmap.

On the other hand, if these folks DO want to use the wxPython GUI (and
all the interactive tools on the canvas) but simply want to be able to
get maps displayed in the canvas by typing commands instead of adding
them to the layer manager, then I think the better way to go is to
enhance the wxPython console. This solution could be of interest to
power users on all platforms.

No matter how much functionality is buit into the wxPython console,
there will still be cases where it would be useful to control the GUI
using external commands (analogous to e.g. "firefox -remote ..." or
gnuclient for XEmacs).

--
Glynn Clements <glynn@gclements.plus.com>

8

Hi,

2009/12/1 Michael Barton <Michael.Barton@asu.edu>:

[...]

On the other hand, if these folks DO want to use the wxPython GUI (and all
the interactive tools on the canvas) but simply want to be able to get maps
displayed in the canvas by typing commands instead of adding them to the
layer manager, then I think the better way to go is to enhance the wxPython
console. This solution could be of interest to power users on all platforms.

It already works, try to type in wxGUI prompt, e.g. `d.rast
map=elevation.dem`. Anyway I think that something like `d.mon wx0`
would be usable.

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa

9

Right you are. I hadn't mentioned this because I thought it wasn't working and didn't have time to look into it. But I only had the wrong syntax.

The d.* commands don't produce a history in the output console, but that is easily changed.

A couple of additional modifications would give this more the feel of the system console.

1) Have the typed input go into the styled text window (output console) instead of a separate command entry box
2) Have the styled text window bind carriage returns so that it will treat any line under the cursor as a "command" to parse when return is pressed.

As Glynn says, this is not exactly the same as a system terminal, but it comes pretty close in functionality.

Michael

On Dec 1, 2009, at 5:54 AM, Martin Landa wrote:

Hi,

2009/12/1 Michael Barton <Michael.Barton@asu.edu>:

[...]

On the other hand, if these folks DO want to use the wxPython GUI (and all
the interactive tools on the canvas) but simply want to be able to get maps
displayed in the canvas by typing commands instead of adding them to the
layer manager, then I think the better way to go is to enhance the wxPython
console. This solution could be of interest to power users on all platforms.

It already works, try to type in wxGUI prompt, e.g. `d.rast
map=elevation.dem`. Anyway I think that something like `d.mon wx0`
would be usable.

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa

10

Attached is a diff that shows how this could be done. It allows you to type a command on the output console, hit return, and have the command run. You can type display commands like d.rast map=elevation.dem and have the map display in the canvas if you have auto-rendering turned on.

As Martin says, this functionality is already in the command entry windows. I just shifted it to the output console too. There are a couple of items that would need to be cleaned up -- hitting return in the middle of a line sends it as a command but also breaks the line. Also, commands entered in this way get echoed twice. But this is just a preliminary example. It should not change any other functionality except for permitting commands to be run from within the output console.

It should be applied to develbranch_6.

I'm not certain that I made the diff correctly (svn diff > file), however. So you might want to look at it first, and try this on a copy.

Michael

(attachments)

console_example.diff (1.64 KB)

11

Here's a somewhat better version of the new command console.

(attachments)

console_example.diff (2.38 KB)

12

2009/12/2 Michael Barton <Michael.Barton@asu.edu>:

Here's a somewhat better version of the new command console.

it should go to prompt.py not goutput.py

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa

13

2009/12/2 Martin Landa <landa.martin@gmail.com>:

2009/12/2 Michael Barton <Michael.Barton@asu.edu>:

Here's a somewhat better version of the new command console.

it should go to prompt.py not goutput.py

probably I am missing something, what is the purpose of your change?

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa

14

An example of how the GUI output console can be modified so that command-line users can use it like a system terminal. Type commands in the terminal, output returned to terminal, have a history of commands and output. But unlike the system terminal, the GUI console captures d.* commands and redirects them to the display canvas.

Functionally, it is little different from what we have now. The difference is that command input is not in a separate window from output.

Michael

On Dec 2, 2009, at 3:40 AM, Martin Landa wrote:

2009/12/2 Martin Landa <landa.martin@gmail.com>:

2009/12/2 Michael Barton <Michael.Barton@asu.edu>:

Here's a somewhat better version of the new command console.

it should go to prompt.py not goutput.py

probably I am missing something, what is the purpose of your change?

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa

15

On Dec 2, 2009, at 3:38 AM, Martin Landa wrote:

2009/12/2 Michael Barton <Michael.Barton@asu.edu>:

Here's a somewhat better version of the new command console.

it should go to prompt.py not goutput.py

I don't understand this suggestion. What should go to prompt.py?

Michael

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa

16

After a night's sleep, this cleans up a couple of annoying issues.

Try it out and see what you think.

Michael

On Dec 1, 2009, at 5:54 AM, Martin Landa wrote:

Hi,

2009/12/1 Michael Barton <Michael.Barton@asu.edu>:

[...]

On the other hand, if these folks DO want to use the wxPython GUI (and all
the interactive tools on the canvas) but simply want to be able to get maps
displayed in the canvas by typing commands instead of adding them to the
layer manager, then I think the better way to go is to enhance the wxPython
console. This solution could be of interest to power users on all platforms.

It already works, try to type in wxGUI prompt, e.g. `d.rast
map=elevation.dem`. Anyway I think that something like `d.mon wx0`
would be usable.

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa

(attachments)

console_example.diff (2.41 KB)

17

2009/12/2 Michael Barton <Michael.Barton@asu.edu>:

I don't understand this suggestion. What should go to prompt.py?

propmt.py defines wxGUI prompt where you type commands.

goutput.py is log area where is printed output of the commands. The
user interaction is not expected here. If understood well, you change
don't follow this precondition.

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa

18

Martin,

I certainly understand that the way the command line interface is currently constructed. The user enters commands into the text box at the bottom and the output is sent to the styled text control at the top.

The discussion has been about how to improve the command-line experience of users who prefer this. I suggested two ways:

1) dispense with the GUI and its code altogether for such users, and just use the system terminal. In that case, render maps to graphics files that are displayed in a simple viewer.
2) enhance the current command interface in the GUI to make it work more like a terminal.

My diff shows how to implement option 2. When entering commands in the system terminal, the same window is used for input and output. There is a scrolling history of commands entered and any text returned as a result of those commands. The diff makes the styled text window--currently used for output only--into such a terminal-like window. Commands can be entered and output received in the same window. The user can place the cursor on any command, hit enter, and run that command regardless of where it is in the window. Commands and output are ordered by when they are executed. Users can copy and paste from any part of the window to any other part. This is sort of like the terminal in R except that it also can run shell commands or any non-interactive command that can be entered from a terminal.

Michael
____________________
C. Michael Barton
Director, Center for Social Dynamics & Complexity
Professor of Anthropology, School of Human Evolution & Social Change
Arizona State University

Phone: 480-965-6262
Fax: 480-965-7671
www: www.public.asu.edu/~cmbarton, http://csdc.asu.edu

On Dec 2, 2009, at 3:12 PM, Martin Landa wrote:

2009/12/2 Michael Barton <Michael.Barton@asu.edu>:

I don't understand this suggestion. What should go to prompt.py?

propmt.py defines wxGUI prompt where you type commands.

goutput.py is log area where is printed output of the commands. The
user interaction is not expected here. If understood well, you change
don't follow this precondition.

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa