[GRASS5] General and gis.m module running (stdout,stderr,exit status) issues

GRASS GIS GRASS GIS developer
1

Hello,

This is a discussion of various issues we are having when running modules and
programs from inside the guis.

1. stderr from module output creeps into the terminal

This was dealt with before by redirecting stderr to /dev/null. We should
really be capturing stderr to display as part of the output. Tcl's open
process commands seem inadequate for this (they can redirect to files and
even open fileids so there might be a solution using pipes). A helper program
that merges these streams together and labels each line or chunk with where
it is from might also be a solution. This might already exist. bash can
easily merge the streams together (can't tell them apart) and add exit status
to the end. Running programs via the unix shell would require proper shell
command line escaping.

2. We can't get exit status from run programs

Tcl's open process commands are inadequate for capturing the exit code of
programs. A helper program that labels this and adds it to the data in stdout
may be in order.

3. In gis.m some menu items, like v.digit, run without a GUI and are useless.

The lowdown:
g.parser eats the original standard input
gis.m is run by g.parser
parser.c decides whether or not to show the UI when there are no arguments
based on whether its standard input is a tty.
When a program is run from inside gis.m parser.c doesn't run the gui because
stdin isn't a tty.

The solution:
Don't run programs inside gis.m and expect to get a GUI. Either use --tcltk to
get the code to source, use --ui to force the user interface to come up, or
run them in a terminal emulator like xterm so that they have a stdin that is
a tty.

Where it's at:
gmmenu.tcl and gm.tcl (The Gm:xmon proc)

On a similar note opening monitors for programs to use and running the program
are independent and should be separated somehow. Rewriting Gm:xmon and
changing the menu entries is probably the best solution.

--Cedric

2

For v.digit, I thought it might be how I tried to work it through gis.m, so
I tried running it from the command line and was still having problems.

Several programs (e.g., v.digit, d.profile) won't run unless there is an
open xmon. The old gis.m tried to help by opening an xmon before starting
one of those programs. People can simply do the whole thing manually, but
this can be complicated for newish users.

One thing I just thought of that might be a good idea in general at this
juncture. For the few modules that require an xterm for display, is there
some way to build in a button (referenced in the C code lines that format
the module gui) that can be used to start an xterm? Or is running anything
like this through parser.c problematic at the moment?

From: Cedric Shock <cedricgrass@shockfamily.net>
Date: Mon, 3 Apr 2006 13:07:34 -0700
To: <grass5@grass.itc.it>
Cc: Michael Barton <michael.barton@asu.edu>
Subject: General and gis.m module running (stdout,stderr,exit status) issues

3. In gis.m some menu items, like v.digit, run without a GUI and are useless.

The lowdown:
g.parser eats the original standard input
gis.m is run by g.parser
parser.c decides whether or not to show the UI when there are no arguments
based on whether its standard input is a tty.
When a program is run from inside gis.m parser.c doesn't run the gui because
stdin isn't a tty.

The solution:
Don't run programs inside gis.m and expect to get a GUI. Either use --tcltk to
get the code to source, use --ui to force the user interface to come up, or
run them in a terminal emulator like xterm so that they have a stdin that is
a tty.

Is this the syntax to put into the menu item? (execute [command] --ui ???)

Where it's at:
gmmenu.tcl and gm.tcl (The Gm:xmon proc)

On a similar note opening monitors for programs to use and running the program
are independent and should be separated somehow. Rewriting Gm:xmon and
changing the menu entries is probably the best solution.

Cheers,
Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics and Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

3

Michael, Markus, Glenn, and everyone,

Is this the syntax to put into the menu item? (execute [command] --ui ???)

execute runs the program with the --tcltk switch and sources the response, so
it will always get a user interface. The syntax to put into the menu item
depends on how we sort out the need for xmons and xterms (see below). Adding
xmon and xterm flags to execute might be a neat syntax like in these
examples:

execute g.region
execute v.digit xmon
execute v.query xmon xterm
execute g.setproj xterm

For v.digit, I thought it might be how I tried to work it through gis.m, so
I tried running it from the command line and was still having problems.

Several programs (e.g., v.digit, d.profile) won't run unless there is an
open xmon. The old gis.m tried to help by opening an xmon before starting
one of those programs. People can simply do the whole thing manually, but
this can be complicated for newish users.

I actually like having gis.m try to open a monitor for them. It just needs to
work, and probably be independent of the method used to run the program. I
think the appropriate logic for opening one would be something like this:

If there's a running selected x display do nothing.
Otherwise start and select the last (or first) not started x display.
If there's no available (not running or running and selected) x display then
do nothing and let the module report an error.

One thing I just thought of that might be a good idea in general at this
juncture. For the few modules that require an xterm for display, is there
some way to build in a button (referenced in the C code lines that format
the module gui) that can be used to start an xterm? Or is running anything
like this through parser.c problematic at the moment?

This is actually done in the tcl code of gui.tcl, not in c. I made a "Run in
xterm" button that's part of the gis.m console ui; it wouldn't be hard to add
one to gui.tcl.

A better approach would be to add something to module descriptions that
requests/demands an interactive terminal. Then gui.tcl and the execute proc
in gis.m could automatically run anything needing an xterm that way instead
of in the "gronsole". This would require a small change to gis.h, parser.c,
g.parser and to every one of those programs to set "needs an interactive
terminal" to true.

Alternatively, and more in line with what you or someone before was doing in
gmmenu.tcl, we could make a list of programs that need an xterm and do all of
this in tcl. This would require a change to SUBMITTING. It would probably
also need some connection to GEM (of which I know nothing) so that extension
modules can report themselves as needing xterms; in this way changing gis.h
etc. might be better. If GEM has commands for menu listings just like ours
then changing execute as described above will solve it all.

The slickest solution of all is to make "gronsole" into a happy terminal
emulator so that everything just works. This is probably outside of what's
feasible for us to do.

Pondering,

--Cedric

4

Cedric,

Alternatively, and more in line with ...
... If GEM has commands for menu listings
just like ours then changing execute as described above will solve it all.

Putting the xterm information in the execute command isn't really enough;
gui.tcl (run via parser.c) wouldn't know in which manner to run a program
that needs an xterm. GEM would certainly need some modification if this
avenue were persued.

Replying to myself in style,

--Cedric

5

Cedric Shock wrote:

This is a discussion of various issues we are having when running modules and
programs from inside the guis.

1. stderr from module output creeps into the terminal

This was dealt with before by redirecting stderr to /dev/null. We should
really be capturing stderr to display as part of the output. Tcl's open
process commands seem inadequate for this (they can redirect to files and
even open fileids so there might be a solution using pipes).

The default is to use a pipe, but you can only do so at one end (i.e.
Tcl can read from the command's output, or write to its input, but not
both).

One issue here is that if the command starts a long-running subprocess
(e.g. (re)starting a driver), Tcl won't get EOF from the pipe until
all of the subprocesses have terminated (i.e. no process has a copy of
the descriptor for the remote end of the pipe).

In practice, this means that the command won't terminate until all of
its descendents have terminated. For a foreground command (with no "&"
at the end of the command), the exec call won't return until this
point.

A concrete example of this causing a problem is the d.resize script,
which stops the driver then restarts it with new settings for
GRASS_WIDTH and GRASS_HEIGHT. A Tcl command of the form
"exec d.resize" won't return until the restarted driver terminates.

The reason that command output was redirected was to prevent this
particular situation.

2. We can't get exit status from run programs

Tcl's open process commands are inadequate for capturing the exit code of
programs. A helper program that labels this and adds it to the data in stdout
may be in order.

Ick. Just live with it until we drop Tcl/Tk in favour of C/C++.

--
Glynn Clements <glynn@gclements.plus.com>

6

Michael Barton wrote:

One thing I just thought of that might be a good idea in general at this
juncture. For the few modules that require an xterm for display, is there
some way to build in a button (referenced in the C code lines that format
the module gui) that can be used to start an xterm? Or is running anything
like this through parser.c problematic at the moment?

The obvious solution is for parser.c to provide a G_terminal_required()
function and for modules which need a terminal to call it before calling
G_parser(). This information can then be made available to gui.tcl.

The other side of the coin is for modules which don't absolutely /need/
a terminal to stop assuming that there is one. I.e. no gratuitous use of
the terminal by modules which are essentially useable without one.

--
Glynn Clements <glynn@gclements.plus.com>

7

On 4/3/06 4:38 PM, "Cedric Shock" <cedricgrass@shockfamily.net> wrote:

execute runs the program with the --tcltk switch and sources the response, so
it will always get a user interface. The syntax to put into the menu item
depends on how we sort out the need for xmons and xterms (see below). Adding
xmon and xterm flags to execute might be a neat syntax like in these
examples:

execute g.region
execute v.digit xmon
execute v.query xmon xterm
execute g.setproj xterm

I like this idea. It's clean and doesn't require anything of the C modules.

For v.digit, I thought it might be how I tried to work it through gis.m, so
I tried running it from the command line and was still having problems.

Several programs (e.g., v.digit, d.profile) won't run unless there is an
open xmon. The old gis.m tried to help by opening an xmon before starting
one of those programs. People can simply do the whole thing manually, but
this can be complicated for newish users.

I actually like having gis.m try to open a monitor for them. It just needs to
work, and probably be independent of the method used to run the program. I
think the appropriate logic for opening one would be something like this:

If there's a running selected x display do nothing.
Otherwise start and select the last (or first) not started x display.
If there's no available (not running or running and selected) x display then
do nothing and let the module report an error.

This is pretty much what my xmon procedure is (or was) doing. So maybe we
just fix this and do what you suggest above?

Michael

___________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

WWW - http://www.public.asu.edu/~cmbarton
Phone: 480-965-6262
Fax: 480-965-7671

8

Sound eminently reasonable.

Michael

On 4/3/06 9:17 PM, "Glynn Clements" <glynn@gclements.plus.com> wrote:

Michael Barton wrote:

One thing I just thought of that might be a good idea in general at this
juncture. For the few modules that require an xterm for display, is there
some way to build in a button (referenced in the C code lines that format
the module gui) that can be used to start an xterm? Or is running anything
like this through parser.c problematic at the moment?

The obvious solution is for parser.c to provide a G_terminal_required()
function and for modules which need a terminal to call it before calling
G_parser(). This information can then be made available to gui.tcl.

The other side of the coin is for modules which don't absolutely /need/
a terminal to stop assuming that there is one. I.e. no gratuitous use of
the terminal by modules which are essentially useable without one.

___________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

WWW - http://www.public.asu.edu/~cmbarton
Phone: 480-965-6262
Fax: 480-965-7671

9

Glynn,

The obvious solution is for parser.c to provide a G_terminal_required()
function and for modules which need a terminal to call it before calling
G_parser(). This information can then be made available to gui.tcl.

There are also bash scripts, which are useful not only for little scripts but
also for using other programs or programming languages. The most natural way
for both c modules and bash modules to convey the "terminal required"
information is via the module definition information. No reason we couldn't
also have a G_terminal_required() that sets this to be true and is the
preferred method of doing so for C code.

The other side of the coin is for modules which don't absolutely /need/
a terminal to stop assuming that there is one. I.e. no gratuitous use of
the terminal by modules which are essentially useable without one.

That'd be nice both from a programming and usability standpoint, but I believe
it is a goal that we won't reach for quite some time.

--Cedric

10

Hmmm, not so sure if I got everything right leading up to
this point.
But GEM itself does not actually interact in any way
with gis.m.
Rather, it works like this (or read the docs that come with GEM):

1. an extension package _may_ contain menu entries for gis.m.
This is a plain ASCII file containing tcl code.
That ASCII file gets copied into $GISBASE/etc/gm/Xtns/

2. gmmenu.tcl sources that tcl snippet into the menu list.

Thus, in whatever way gis.m will handle xterm etc., it's
just a matter of updating one ASCII file in the extension package
to produce the right code.

I think ...

Cedric Shock wrote:

Cedric,

Alternatively, and more in line with ...
... If GEM has commands for menu listings just like ours then changing execute as described above will solve it all.

Putting the xterm information in the execute command isn't really enough; gui.tcl (run via parser.c) wouldn't know in which manner to run a program that needs an xterm. GEM would certainly need some modification if this avenue were persued.

Replying to myself in style,

--Cedric

_______________________________________________
grass5 mailing list
grass5@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass5

11

On Tue, 4 Apr 2006 05:12:53 +0100
Glynn Clements <glynn@gclements.plus.com> wrote:

Cedric Shock wrote:

<SNIP>

> 2. We can't get exit status from run programs
>
> Tcl's open process commands are inadequate for capturing the exit
> code of programs. A helper program that labels this and adds it to
> the data in stdout may be in order.

Ick. Just live with it until we drop Tcl/Tk in favour of C/C++.

Glynn, Cedric,

Are you here talking about this issue:
http://intevation.de/rt/webrt?serial_num=3010

I'd like to add your commnets into the tracker if so.

Maciek

--------------------
W polskim Internecie s? setki milion?w stron. My przekazujemy Tobie tylko najlepsze z nich!
http://katalog.panoramainternetu.pl/

12

Maciek Sieczka wrote:

> > 2. We can't get exit status from run programs
> >
> > Tcl's open process commands are inadequate for capturing the exit
> > code of programs. A helper program that labels this and adds it to
> > the data in stdout may be in order.
>
> Ick. Just live with it until we drop Tcl/Tk in favour of C/C++.

Glynn, Cedric,

Are you here talking about this issue:
http://intevation.de/rt/webrt?serial_num=3010

No; the issue here is that you can't get the process' return code (the
value passed to exit() or returned from main()) from within Tcl.

--
Glynn Clements <glynn@gclements.plus.com>