[GRASS5] Need testers for tcltkgrass update for 5.7

Glynn Clements wrote:

I replaced most commands of the form "exec <cmd> &" with "execute
cmd". This only applies where the command was called without arguments
(where the command will typically call G_gui()). Ultimately, I suspect
that you will need to provide more information than just the name of
the command.

I installed your new version, but now, all those commands that still use
"exec" are missing the "&" and block the entire tcltkgrass menu while they
are active. When I click on "close", I get an error message (example of
r.sunmask):

child process exited abnormally
child process exited abnormally
    while executing
"exec r.sunmask"
    invoked from within
".main_menu.mb3.m.m9 invoke active"
    ("uplevel" body line 1)
etc...

This is also true for the d.mon commands which makes them unusable with
the tcltkgrass menu.

Moritz

Moritz Lennert wrote:

> I replaced most commands of the form "exec <cmd> &" with "execute
> cmd". This only applies where the command was called without arguments
> (where the command will typically call G_gui()). Ultimately, I suspect
> that you will need to provide more information than just the name of
> the command.

I installed your new version, but now, all those commands that still use
"exec" are missing the "&" and block the entire tcltkgrass menu while they
are active.

The execute procedure is defined at the top of gui.tcl as:

  proc execute {cmd} {
      exec -- $cmd &
  }

Anything which is run via "execute" should get run in the background.

When I click on "close", I get an error message (example of
r.sunmask):

Hmm. For some reason, a few commands are using exec (without the &)
rather than execute; r.sunmask is one of them.

AFAICT, the list of incorrect cases is:

  r.out.gridatb
  g3.createwind
  g3.setregion
  g.setproj
  d.monsize
  r.sunmask
  r.shaded.relief
  r.le.pixel
  r.le.patch
  r.le.trace
  v.segment

I'm not sure how this occurred; I've attached an update.

This is also true for the d.mon commands which makes them unusable with
the tcltkgrass menu.

The d.mon commands are run with arguments. They should complete more
or less immediately, and so shouldn't need to be run in the
background.

--
Glynn Clements <glynn.clements@virgin.net>

(attachments)

menu.tcl.gz (6.35 KB)

Hmm. For some reason, a few commands are using exec (without the &)
rather than execute; r.sunmask is one of them.

AFAICT, the list of incorrect cases is:

  r.out.gridatb
  g3.createwind
  g3.setregion
  g.setproj
  d.monsize
  r.sunmask
  r.shaded.relief
  r.le.pixel
  r.le.patch
  r.le.trace
  v.segment

I'm not sure how this occurred; I've attached an update.

Thanks, now it works fine for these commands.

This is also true for the d.mon commands which makes them unusable with
the tcltkgrass menu.

The d.mon commands are run with arguments. They should complete more
or less immediately, and so shouldn't need to be run in the
background.

I have the feeling the problem might be the output of the d.mon command.
The actual tcl error message I get is:

using default visual which is TrueColor
ncolors: 16777216
Graphics driver [x0] started
using default visual which is TrueColor
ncolors: 16777216
Graphics driver [x0] started
    while executing
"exec d.mon start=x0"

Maybe tcl cannot handle the output (first two lines of the error message) ?

Moritz

Moritz Lennert wrote:

> The d.mon commands are run with arguments. They should complete more
> or less immediately, and so shouldn't need to be run in the
> background.

I have the feeling the problem might be the output of the d.mon command.
The actual tcl error message I get is:

using default visual which is TrueColor
ncolors: 16777216
Graphics driver [x0] started
using default visual which is TrueColor
ncolors: 16777216
Graphics driver [x0] started
    while executing
"exec d.mon start=x0"

Maybe tcl cannot handle the output (first two lines of the error message) ?

Ok; the exec(n) manpage says:

  If any of the commands writes to its standard error file and
  that standard error isn't redirected, then exec will return an
  error

So, they should probably be run with e.g.:

  exec d.mon start=x0 2> /dev/null
or:
  exec d.mon start=x0 2>@ stdout

Again, this form of command execution should probably be wrapped up
into a procedure, e.g.:

  proc run {cmd args} {
    eval exec -- $cmd $args 2>@ stdout
  }

[However, there's already a procedure named "run", which starts a
command with its own xterm; that's currently only used for
r.le.setup.]

Essentially, we at least need one procedure for executing commands
which will end up calling G_gui() (so that we only have one place in
tcltkgrass which needs to be changed if we were to make G_gui() send
the Tcl/Tk code back to tcltkgrass for execution) and another one for
non-interactive commands such as "d.mon start=...". For now, anything
which doesn't fit those two cases can just use Tcl's exec directly.

--
Glynn Clements <glynn.clements@virgin.net>

Michael Barton wrote:

Some of the commands Glynn lists below can be run with his new execute
procedure, which automates running a command modally. As he says, commands
with arguments much be run the old way with "exec $cmd $arg &". This does
include d.mon commands. If they are not run modally, you have no further
access to the menu for anything.

I figured out the problem with the d.mon commands. Because stdout
wasn't redirected, exec tried to read from it until EOF to return that
data as its result. "d.mon start=..." forks the driver, so exec
doesn't get EOF until the driver dies (when a pipe is shared by
multiple processes, the reader won't receive EOF until *all* writers
have closed the write end). BTW, this shouldn't apply to stop/select,
only to start.

There are several other commands, including a few not listed here, which
still need to run in a separate xterm (r.digit, for example). I've also
fixed these so that they run properly.

The attached version no longer directly uses exec for anything.
Instead, everything goes through one of four functions:

  execute: runs a command with no arguments in the background;
  this function is intended solely for cases which will end up
  in G_gui().

  spawn: runs a command with arguments in the background (so
  stdout/stderr go to tcltkgrass' stdout/stderr, i.e. the
  terminal from which it was started).

  run: runs a command with arguments in the foreground, with
  stdout/stderr explicitly redirected to tcltkgrass'
  stdout/stderr (to prevent cases such as "d.mon start=..." from
  hanging).

  term: runs a command with arguments on its own xterm, in the
  background.

Ultimately, "run" should probably be split further. Many commands
probably *shouldn't* have stdout/stderr redirected, but should return
this information via exec to be displayed by tcltkgrass (e.g. the
g.version option should display a dialog rather than dumping the data
on the terminal from which tcltkgrass was run). Obviously, we can't do
that for "d.mon start=...". Although maybe the ideal solution there is
to make the drivers behave more like daemons, i.e. close their
standard descriptors, start a new session (in the setsid() sense) and
disassociate from the controlling terminal.

Also, the xterm case isn't trivial to get right. If xterm is setuid
(as is the case on many systems, including mine), LD_LIBRARY_PATH is
reset (as a security measure), typically causing shared GRASS
libraries to become inaccessible. We really need a "grass-run" script
which will set LD_LIBRARY_PATH back to the proper value before running
the specified command.

Additional changes include removing references to:

scripting, since it doesn't work anymore

It doesn't work currently; it isn't out of the question to get it
working in the future.

saving configurations, since that doesn't work anymore (save d.m files
instead)
configuring netscape, also doesn't work

BTW, I know that this isn't a tcltkgrass issue, but the default
GRASS_HTML_BROWSER settings need some work. E.g. they typically don't
do the right thing if you already have the browser open.

g3.setregion and g3.setwind don't work because these scripts are not yet
g.parser() compliant. Unless there is a reason NOT to do this, I will go
ahead and add the headers to make them autogenerate a GRASS 5.7 GUI dialog.

Sure. In the meantime, I changed these to use execute as well. Right
now, nothing will work, as they can't won't work without arguments.
Once they've been g.parser-ised, execute is the right thing to use.

Glynn: the new procedure set_menu_font doesn't work for some reason. It
looks like it should, but when I call it from the config menu it says
main_menu(font) not found or something to that nature. So I reverted to the
previous way of doing this and it works again.

Duh; I forgot to put "global main_menu" in the procedure.

--
Glynn Clements <glynn.clements@virgin.net>

(attachments)

tcltkgrass_test.tar.gz (10.2 KB)

Thanks Glynn,

This is a big help in updating tcltkgrass for GRASS 5.7.

I'll check this version on my Mac also and look it over. I'll wait to commit
this until I hear back from Moritz (or anyone else who wants to weigh in on
this). I need to understand which commands need which of the spawn, run, and
term procedures and why.

Ideally, it would be nice if there we only needed 2 ways to run a GRASS
command from tcltkgrass: 1) the command runs via a G_gui() autogenerated
dialog and/or output is returned to a tcltk window; and 2) the command runs
in an xterminal (distinct from the GRASS shell terminal) and output is
directed to the same terminal. Currently, there are only a small percentage
of the GRASS commands that fall into category 2. The rest work fine with the
execute procedure. Can the execute procedure be augmented to permit commands
with arguments that normally run via G_gui to be processed? This would leave
only the tiny fraction that still require an xterm to be dealt with.

Michael

On 8/17/04 4:23 PM, "Glynn Clements" <glynn.clements@virgin.net> wrote:

Michael Barton wrote:

Some of the commands Glynn lists below can be run with his new execute
procedure, which automates running a command modally. As he says, commands
with arguments much be run the old way with "exec $cmd $arg &". This does
include d.mon commands. If they are not run modally, you have no further
access to the menu for anything.

I figured out the problem with the d.mon commands. Because stdout
wasn't redirected, exec tried to read from it until EOF to return that
data as its result. "d.mon start=..." forks the driver, so exec
doesn't get EOF until the driver dies (when a pipe is shared by
multiple processes, the reader won't receive EOF until *all* writers
have closed the write end). BTW, this shouldn't apply to stop/select,
only to start.

There are several other commands, including a few not listed here, which
still need to run in a separate xterm (r.digit, for example). I've also
fixed these so that they run properly.

The attached version no longer directly uses exec for anything.
Instead, everything goes through one of four functions:

execute: runs a command with no arguments in the background;
this function is intended solely for cases which will end up
in G_gui().

spawn: runs a command with arguments in the background (so
stdout/stderr go to tcltkgrass' stdout/stderr, i.e. the
terminal from which it was started).

run: runs a command with arguments in the foreground, with
stdout/stderr explicitly redirected to tcltkgrass'
stdout/stderr (to prevent cases such as "d.mon start=..." from
hanging).

term: runs a command with arguments on its own xterm, in the
background.

Ultimately, "run" should probably be split further. Many commands
probably *shouldn't* have stdout/stderr redirected, but should return
this information via exec to be displayed by tcltkgrass (e.g. the
g.version option should display a dialog rather than dumping the data
on the terminal from which tcltkgrass was run). Obviously, we can't do
that for "d.mon start=...". Although maybe the ideal solution there is
to make the drivers behave more like daemons, i.e. close their
standard descriptors, start a new session (in the setsid() sense) and
disassociate from the controlling terminal.

Also, the xterm case isn't trivial to get right. If xterm is setuid
(as is the case on many systems, including mine), LD_LIBRARY_PATH is
reset (as a security measure), typically causing shared GRASS
libraries to become inaccessible. We really need a "grass-run" script
which will set LD_LIBRARY_PATH back to the proper value before running
the specified command.

Additional changes include removing references to:

scripting, since it doesn't work anymore

It doesn't work currently; it isn't out of the question to get it
working in the future.

saving configurations, since that doesn't work anymore (save d.m files
instead)
configuring netscape, also doesn't work

BTW, I know that this isn't a tcltkgrass issue, but the default
GRASS_HTML_BROWSER settings need some work. E.g. they typically don't
do the right thing if you already have the browser open.

g3.setregion and g3.setwind don't work because these scripts are not yet
g.parser() compliant. Unless there is a reason NOT to do this, I will go
ahead and add the headers to make them autogenerate a GRASS 5.7 GUI dialog.

Sure. In the meantime, I changed these to use execute as well. Right
now, nothing will work, as they can't won't work without arguments.
Once they've been g.parser-ised, execute is the right thing to use.

Glynn: the new procedure set_menu_font doesn't work for some reason. It
looks like it should, but when I call it from the config menu it says
main_menu(font) not found or something to that nature. So I reverted to the
previous way of doing this and it works again.

Duh; I forgot to put "global main_menu" in the procedure.

--
Glynn Clements <glynn.clements@virgin.net>

______________________________
Michael Barton, Professor & Curator
School of Human Origins, Cultures, & Societies
Arizona State University
Tempe, AZ 85287-2402
USA

voice: 480-965-6262; fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

Michael Barton wrote:

I'll check this version on my Mac also and look it over. I'll wait to commit
this until I hear back from Moritz (or anyone else who wants to weigh in on
this). I need to understand which commands need which of the spawn, run, and
term procedures and why.

Well, in general, anything which performs interaction on the terminal
needs to use "term", anything else which runs indefinitely needs to
use "spawn", and the rest (i.e. anything which isn't interactive and
which doesn't run indefinitely) can use "run".

The "runs indefinitely" is actually quite tricky. Primarily because
running multiple GRASS programs concurrently is problematic. If you
start a long-running program such as v.digit, it's debatable whether
you should be allowed to run anything else until it finishes.

From the perspective of neatness, it would be better for the UI to
remain functional, but to inhibit the execution of new commands
(except those which are always safe, e.g. g.manual) than to simply
"hang" until the command completes. However, implementing that would
be non-trivial; you would need to start all commands in the
background, then manually check for termination (and I have no idea
how to do that in Tcl).

And the ideal, i.e. only allowing further commands to be run if they
are "compatible" with outstanding commands is, I suspect, so complex
as to be practically impossible.

Ideally, it would be nice if there we only needed 2 ways to run a GRASS
command from tcltkgrass: 1) the command runs via a G_gui() autogenerated
dialog and/or output is returned to a tcltk window; and 2) the command runs
in an xterminal (distinct from the GRASS shell terminal) and output is
directed to the same terminal.

That doesn't account for the situation where the command needs no
arguments (or the menu entry includes specific arguments, e.g. the
start/stop/select monitors entries) and performs no interaction. I
suppose that you could force it into the G_gui() mould, i.e. a dialog
with zero arguments.

Currently, there are only a small percentage
of the GRASS commands that fall into category 2. The rest work fine with the
execute procedure. Can the execute procedure be augmented to permit commands
with arguments that normally run via G_gui to be processed?

AFAICT, if you supply one or more arguments on the command-line,
G_gui() (or the old interactive() if GRASS_UI_TERM is set) is never
called. So I can't see any point in allowing the "execute" procedure
to accept arguments (it would be trivial to support, though).

That reminds me; there's a subtle issue regarding d.what.*: if one or
more maps (of the correct type) are displayed on the monitor, it won't
call into G_gui() even if no arguments are supplied. Whilst this is
convenient if this was what you actually wanted, it essentially means
that it won't offer you the option to provide arguments via a dialog.

A similar issue applies to commands for which all arguments are
optional (e.g. d.erase). I guess that we need a switch (or environment
variable) which will force the use of G_gui() even when a command can
be run with no arguments.

--
Glynn Clements <glynn.clements@virgin.net>

Glynn,

Thanks for the very useful input. I tried your modification and they work
just fine under Mac OS X. Looking at your new procedures for calling
programs and doing a bit of experimentation has led to a few questions.

On 8/17/04 5:43 PM, "Glynn Clements" <glynn.clements@virgin.net> wrote:

Michael Barton wrote:

I'll check this version on my Mac also and look it over. I'll wait to commit
this until I hear back from Moritz (or anyone else who wants to weigh in on
this). I need to understand which commands need which of the spawn, run, and
term procedures and why.

Well, in general, anything which performs interaction on the terminal
needs to use "term", anything else which runs indefinitely needs to
use "spawn", and the rest (i.e. anything which isn't interactive and
which doesn't run indefinitely) can use "run".

OK. I understand this distinction and can see what the differences are
between the execute, spawn, and run procedures.

But now I suppose I betray my ignorance. If I call a normal GRASS 5.7
command with no argument (i.e., it is parsed by G_gui() and autogenerates an
interactive tcltk dialog) through either the spawn or run procedures it
*appears* to work the same as if I called it via the execute procedure. Is
there a problem in doing this? Why can't I just use run, for example, to
call all GRASS programs except those that require an xterm?

The "runs indefinitely" is actually quite tricky. Primarily because
running multiple GRASS programs concurrently is problematic. If you
start a long-running program such as v.digit, it's debatable whether
you should be allowed to run anything else until it finishes.

From the perspective of neatness, it would be better for the UI to
remain functional, but to inhibit the execution of new commands
(except those which are always safe, e.g. g.manual) than to simply
"hang" until the command completes. However, implementing that would
be non-trivial; you would need to start all commands in the
background, then manually check for termination (and I have no idea
how to do that in Tcl).

And the ideal, i.e. only allowing further commands to be run if they
are "compatible" with outstanding commands is, I suspect, so complex
as to be practically impossible.

Is the possibility of running incompatible commands really a potential
problem? If so, maybe there is a way around it and satisfying at least some
of the conditions you mention above. I've not yet run into this kind of
incompatibility probably because in actually working with GRASS, I often
'run' several commands, but then leave them 'paused' at the opening dialog.
I then switch back and forth between commands so that I am actually running
only a single command at a time, rather than running them concurrently in
the full sense. I suspect that this is what most people do most of the time.

Under such circumstances, could pressing the 'run' button on the dialog box
also inhibit the execution of other commands except for 'safe' ones? If so,
this would work for most commands, I think, without compromising the
functionality of the UI. Notable exceptions would be d.m, d.mon, and nviz.

Ideally, it would be nice if there we only needed 2 ways to run a GRASS
command from tcltkgrass: 1) the command runs via a G_gui() autogenerated
dialog and/or output is returned to a tcltk window; and 2) the command runs
in an xterminal (distinct from the GRASS shell terminal) and output is
directed to the same terminal.

That doesn't account for the situation where the command needs no
arguments (or the menu entry includes specific arguments, e.g. the
start/stop/select monitors entries) and performs no interaction. I
suppose that you could force it into the G_gui() mould, i.e. a dialog
with zero arguments.

Currently, there are only a small percentage
of the GRASS commands that fall into category 2. The rest work fine with the
execute procedure. Can the execute procedure be augmented to permit commands
with arguments that normally run via G_gui to be processed?

AFAICT, if you supply one or more arguments on the command-line,
G_gui() (or the old interactive() if GRASS_UI_TERM is set) is never
called. So I can't see any point in allowing the "execute" procedure
to accept arguments (it would be trivial to support, though).

Hmm. But it seems that commands without arguments can 'run' or 'spawn' like
commands that need arguments and still manage to go through G_gui(). Given
this, is there still a need for the 'execute' procedure?

If I call a GRASS command via the command line shell, it is somehow parsed
correctly--going to a G_gui() dialog or an interactive xterm if there are no
arguments, or performing its function if there are arguments. Is it a
limitation of tcltk that prevents this parser (i.e., that correctly routes
different kinds of calls from the command line) from operating in the same
way in this context? Sorry if I'm being dense, but this discussion has been
very helpful in better understanding how this operates.

That reminds me; there's a subtle issue regarding d.what.*: if one or
more maps (of the correct type) are displayed on the monitor, it won't
call into G_gui() even if no arguments are supplied. Whilst this is
convenient if this was what you actually wanted, it essentially means
that it won't offer you the option to provide arguments via a dialog.

Double hmm. I'd never noticed this before. Perhaps this issue is connected
to the reason that d.what.vect give a 'broken pipe' error after the 2nd time
it is run in a GRASS session unless you force it to xterm mode with -x.

A similar issue applies to commands for which all arguments are
optional (e.g. d.erase). I guess that we need a switch (or environment
variable) which will force the use of G_gui() even when a command can
be run with no arguments.

It would indeed be nice to have greater consistency in how commands need to
be called.

Thanks again for your help.

Michael
____________________
C. Michael Barton, Professor
School of Human Origins, Cultures, & Societies
PO Box 872402
Arizona State University
Tempe, AZ 85287-2402
USA

Phone: 480-965-6262
Fax: 480-965-7671
www: <www.public.asu.edu/~cmbarton>

Thanks Glynn,

This is a big help in updating tcltkgrass for GRASS 5.7.

I'll check this version on my Mac also and look it over. I'll wait to
commit
this until I hear back from Moritz (or anyone else who wants to weigh in
on
this).

Seems to work perfectly for me.

Moritz

Michael Barton wrote:

>> I'll check this version on my Mac also and look it over. I'll wait to commit
>> this until I hear back from Moritz (or anyone else who wants to weigh in on
>> this). I need to understand which commands need which of the spawn, run, and
>> term procedures and why.
>
> Well, in general, anything which performs interaction on the terminal
> needs to use "term", anything else which runs indefinitely needs to
> use "spawn", and the rest (i.e. anything which isn't interactive and
> which doesn't run indefinitely) can use "run".

OK. I understand this distinction and can see what the differences are
between the execute, spawn, and run procedures.

But now I suppose I betray my ignorance. If I call a normal GRASS 5.7
command with no argument (i.e., it is parsed by G_gui() and autogenerates an
interactive tcltk dialog) through either the spawn or run procedures it
*appears* to work the same as if I called it via the execute procedure. Is
there a problem in doing this? Why can't I just use run, for example, to
call all GRASS programs except those that require an xterm?

Future expansion.

Essentially, I want to allow for the possibility of adding a feature
to G_parser/G_gui such that, when the program is being run from
tcltkgrass, rather than G_gui() generating Tcl/Tk code and feeding it
directly to wish, it would write it to stdout, and tcltkgrass would
read and execute the generated code.

This would allow tcltkgrass to implement features such as scripting,
command history, etc, as tcltkgrass would get to see the user's
argument values.

Implementing this would essentially boil down to:

1. Adding an option to G_gui() to write the code to stdout rather than
calling popen("$GRASS_WISH").

2. Adding a global switch (e.g. --tcltk) to G_parser(), which makes
use of 1.

3. Copying the relevant portions of lib/gis/gui.tcl into tcltkgrass'
gui.tcl, with any necessary modifications.

4. Changing the execute procedure to e.g.:

  proc execute {cmd} {
      eval [exec -- $cmd --tcltk 2>@ stderr]
  }

So, execute would simply invoke the program to get the Tcl/Tk code for
the dialog, then evaluate it. The actual execution of the program
(with arguments) would occur when the user clicked the dialog's "Run"
button.

In short, while execute currently looks very much like spawn,
ultimately it could look very different.

> The "runs indefinitely" is actually quite tricky. Primarily because
> running multiple GRASS programs concurrently is problematic. If you
> start a long-running program such as v.digit, it's debatable whether
> you should be allowed to run anything else until it finishes.
>
> From the perspective of neatness, it would be better for the UI to
> remain functional, but to inhibit the execution of new commands
> (except those which are always safe, e.g. g.manual) than to simply
> "hang" until the command completes. However, implementing that would
> be non-trivial; you would need to start all commands in the
> background, then manually check for termination (and I have no idea
> how to do that in Tcl).
>
> And the ideal, i.e. only allowing further commands to be run if they
> are "compatible" with outstanding commands is, I suspect, so complex
> as to be practically impossible.

Is the possibility of running incompatible commands really a potential
problem? If so, maybe there is a way around it and satisfying at least some
of the conditions you mention above. I've not yet run into this kind of
incompatibility probably because in actually working with GRASS, I often
'run' several commands, but then leave them 'paused' at the opening dialog.
I then switch back and forth between commands so that I am actually running
only a single command at a time, rather than running them concurrently in
the full sense. I suspect that this is what most people do most of the time.

OK, that usage isn't an issue. It's only when the commands are
actually "running" (i.e. modifying the mapset directory) that the
issues arise.

Under such circumstances, could pressing the 'run' button on the dialog box
also inhibit the execution of other commands except for 'safe' ones? If so,
this would work for most commands, I think, without compromising the
functionality of the UI. Notable exceptions would be d.m, d.mon, and nviz.

That would work; however:

1. It would require that tcltkgrass was managing the dialogs, as
described above.

2. To actually implement mutual exclusion, you would need to either:

a) have the Run button execute the command in the foreground (without
the trailing "&"), or

b) find some way of determining when a background command has
completed.

The problem with 1 is that the UI will remain unresponsive (it won't
even redraw itself) while the command runs, which is ugly.

The problem with 2 is that I don't know how to do this in Tcl. When
the last argument to exec is "&", it returns the process identifier(s)
for the command, but I don't see a Tcl version of waitpid().

> AFAICT, if you supply one or more arguments on the command-line,
> G_gui() (or the old interactive() if GRASS_UI_TERM is set) is never
> called. So I can't see any point in allowing the "execute" procedure
> to accept arguments (it would be trivial to support, though).

Hmm. But it seems that commands without arguments can 'run' or 'spawn' like
commands that need arguments and still manage to go through G_gui(). Given
this, is there still a need for the 'execute' procedure?

If I call a GRASS command via the command line shell, it is somehow parsed
correctly--going to a G_gui() dialog or an interactive xterm if there are no
arguments, or performing its function if there are arguments. Is it a
limitation of tcltk that prevents this parser (i.e., that correctly routes
different kinds of calls from the command line) from operating in the same
way in this context? Sorry if I'm being dense, but this discussion has been
very helpful in better understanding how this operates.

See my comments regarding "execute" above.

I initially described the idea a couple of days ago in response to
your comment about scripting not working:

: From: Glynn Clements <glynn.clements@virgin.net>
: Subject: Re: [GRASS5] Need testers for tcltkgrass update for 5.7
: Date: Mon, 16 Aug 2004 14:41:32 +0100
: Message-ID: <16672.47500.961075.166905@cerise.nosuchdomain.co.uk>
:
: > The only thing that doesn't work is scripting. I'm not sure whether this is
: > a function of GRASS 5.7 (and that I should take it out of the menu
: > altogether) or it is simply broken at the moment. If anyone has a clue, let
: > me know.
:
: The old tcltkgrass constructed and handled the dialogs itself, so it
: got to see the actual command line. The new version simply runs the
: command without arguments; the command generates the dialog, reads the
: arguments, then re-executes itself, without tcltkgrass ever getting to
: see the arguments.
:
: To make tcltkgrass scripting work, G_gui() would need to e.g. write
: the Tcl/Tk code to stdout so that it could be executed by tcltkgrass.
: I.e. something like --html-description but which generates Tcl/Tk code
: rather than HTML.

However, I didn't really make clear the connection between this and
the "execute" procedure.

--
Glynn Clements <glynn.clements@virgin.net>

Glynn,

This sounds like a plan. I'll go ahead and commit menu.tcl and gui.tcl it as
is now (Moritz had no problems with it either). I think this kind of
scripting within GRASS is potentially quite useful. I wonder how many people
use it in GRASS 5.3? To revive scripting, all commands would have to
eventually be set up to write to stdout, correct?

Michael

On 8/18/04 7:04 AM, "Glynn Clements" <glynn.clements@virgin.net> wrote:

Michael Barton wrote:

I'll check this version on my Mac also and look it over. I'll wait to
commit
this until I hear back from Moritz (or anyone else who wants to weigh in on
this). I need to understand which commands need which of the spawn, run,
and
term procedures and why.

Well, in general, anything which performs interaction on the terminal
needs to use "term", anything else which runs indefinitely needs to
use "spawn", and the rest (i.e. anything which isn't interactive and
which doesn't run indefinitely) can use "run".

OK. I understand this distinction and can see what the differences are
between the execute, spawn, and run procedures.

But now I suppose I betray my ignorance. If I call a normal GRASS 5.7
command with no argument (i.e., it is parsed by G_gui() and autogenerates an
interactive tcltk dialog) through either the spawn or run procedures it
*appears* to work the same as if I called it via the execute procedure. Is
there a problem in doing this? Why can't I just use run, for example, to
call all GRASS programs except those that require an xterm?

Future expansion.

Essentially, I want to allow for the possibility of adding a feature
to G_parser/G_gui such that, when the program is being run from
tcltkgrass, rather than G_gui() generating Tcl/Tk code and feeding it
directly to wish, it would write it to stdout, and tcltkgrass would
read and execute the generated code.

This would allow tcltkgrass to implement features such as scripting,
command history, etc, as tcltkgrass would get to see the user's
argument values.

Implementing this would essentially boil down to:

1. Adding an option to G_gui() to write the code to stdout rather than
calling popen("$GRASS_WISH").

2. Adding a global switch (e.g. --tcltk) to G_parser(), which makes
use of 1.

3. Copying the relevant portions of lib/gis/gui.tcl into tcltkgrass'
gui.tcl, with any necessary modifications.

4. Changing the execute procedure to e.g.:

proc execute {cmd} {
   eval [exec -- $cmd --tcltk 2>@ stderr]
}

So, execute would simply invoke the program to get the Tcl/Tk code for
the dialog, then evaluate it. The actual execution of the program
(with arguments) would occur when the user clicked the dialog's "Run"
button.

In short, while execute currently looks very much like spawn,
ultimately it could look very different.

The "runs indefinitely" is actually quite tricky. Primarily because
running multiple GRASS programs concurrently is problematic. If you
start a long-running program such as v.digit, it's debatable whether
you should be allowed to run anything else until it finishes.

From the perspective of neatness, it would be better for the UI to
remain functional, but to inhibit the execution of new commands
(except those which are always safe, e.g. g.manual) than to simply
"hang" until the command completes. However, implementing that would
be non-trivial; you would need to start all commands in the
background, then manually check for termination (and I have no idea
how to do that in Tcl).

And the ideal, i.e. only allowing further commands to be run if they
are "compatible" with outstanding commands is, I suspect, so complex
as to be practically impossible.

Is the possibility of running incompatible commands really a potential
problem? If so, maybe there is a way around it and satisfying at least some
of the conditions you mention above. I've not yet run into this kind of
incompatibility probably because in actually working with GRASS, I often
'run' several commands, but then leave them 'paused' at the opening dialog.
I then switch back and forth between commands so that I am actually running
only a single command at a time, rather than running them concurrently in
the full sense. I suspect that this is what most people do most of the time.

OK, that usage isn't an issue. It's only when the commands are
actually "running" (i.e. modifying the mapset directory) that the
issues arise.

Under such circumstances, could pressing the 'run' button on the dialog box
also inhibit the execution of other commands except for 'safe' ones? If so,
this would work for most commands, I think, without compromising the
functionality of the UI. Notable exceptions would be d.m, d.mon, and nviz.

That would work; however:

1. It would require that tcltkgrass was managing the dialogs, as
described above.

2. To actually implement mutual exclusion, you would need to either:

a) have the Run button execute the command in the foreground (without
the trailing "&"), or

b) find some way of determining when a background command has
completed.

The problem with 1 is that the UI will remain unresponsive (it won't
even redraw itself) while the command runs, which is ugly.

The problem with 2 is that I don't know how to do this in Tcl. When
the last argument to exec is "&", it returns the process identifier(s)
for the command, but I don't see a Tcl version of waitpid().

AFAICT, if you supply one or more arguments on the command-line,
G_gui() (or the old interactive() if GRASS_UI_TERM is set) is never
called. So I can't see any point in allowing the "execute" procedure
to accept arguments (it would be trivial to support, though).

Hmm. But it seems that commands without arguments can 'run' or 'spawn' like
commands that need arguments and still manage to go through G_gui(). Given
this, is there still a need for the 'execute' procedure?

If I call a GRASS command via the command line shell, it is somehow parsed
correctly--going to a G_gui() dialog or an interactive xterm if there are no
arguments, or performing its function if there are arguments. Is it a
limitation of tcltk that prevents this parser (i.e., that correctly routes
different kinds of calls from the command line) from operating in the same
way in this context? Sorry if I'm being dense, but this discussion has been
very helpful in better understanding how this operates.

See my comments regarding "execute" above.

I initially described the idea a couple of days ago in response to
your comment about scripting not working:

: From: Glynn Clements <glynn.clements@virgin.net>
: Subject: Re: [GRASS5] Need testers for tcltkgrass update for 5.7
: Date: Mon, 16 Aug 2004 14:41:32 +0100
: Message-ID: <16672.47500.961075.166905@cerise.nosuchdomain.co.uk>
:
: > The only thing that doesn't work is scripting. I'm not sure whether this
is
: > a function of GRASS 5.7 (and that I should take it out of the menu
: > altogether) or it is simply broken at the moment. If anyone has a clue,
let
: > me know.
:
: The old tcltkgrass constructed and handled the dialogs itself, so it
: got to see the actual command line. The new version simply runs the
: command without arguments; the command generates the dialog, reads the
: arguments, then re-executes itself, without tcltkgrass ever getting to
: see the arguments.
:
: To make tcltkgrass scripting work, G_gui() would need to e.g. write
: the Tcl/Tk code to stdout so that it could be executed by tcltkgrass.
: I.e. something like --html-description but which generates Tcl/Tk code
: rather than HTML.

However, I didn't really make clear the connection between this and
the "execute" procedure.

--
Glynn Clements <glynn.clements@virgin.net>

______________________________
Michael Barton, Professor & Curator
School of Human Origins, Cultures, & Societies
Arizona State University
Tempe, AZ 85287-2402
USA

voice: 480-965-6262; fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

Michael Barton wrote:

This sounds like a plan. I'll go ahead and commit menu.tcl and gui.tcl it as
is now (Moritz had no problems with it either). I think this kind of
scripting within GRASS is potentially quite useful. I wonder how many people
use it in GRASS 5.3? To revive scripting, all commands would have to
eventually be set up to write to stdout, correct?

Well, the change is essentially to G_parser() and G_gui(), so it would
automatically affect everything which uses those functions.

However, fixing startup bugs/misfeatures (e.g. ensuring that programs
don't write to stdout prior to calling G_parser()) would become that
much more critical. It's already an issue for e.g. --html-description,
but bugs in generating the HTML documentation are less critical than a
bug in the UI.

Also, programs which don't use G_parser() (e.g. r.mapcalc) will
typically need special treatment (e.g. r.mapcalc explicitly traps the
"r.mapcalc help" case, although it doesn't handle any of the others).
Maybe we need a "minimal" alternative to G_parser(), which just traps
--help, --html-description etc, but doesn't do option parsing.

--
Glynn Clements <glynn.clements@virgin.net>

Glynn Clements wrote:

> But now I suppose I betray my ignorance. If I call a normal GRASS 5.7
> command with no argument (i.e., it is parsed by G_gui() and autogenerates an
> interactive tcltk dialog) through either the spawn or run procedures it
> *appears* to work the same as if I called it via the execute procedure. Is
> there a problem in doing this? Why can't I just use run, for example, to
> call all GRASS programs except those that require an xterm?

Future expansion.

Essentially, I want to allow for the possibility of adding a feature
to G_parser/G_gui such that, when the program is being run from
tcltkgrass, rather than G_gui() generating Tcl/Tk code and feeding it
directly to wish, it would write it to stdout, and tcltkgrass would
read and execute the generated code.

This would allow tcltkgrass to implement features such as scripting,
command history, etc, as tcltkgrass would get to see the user's
argument values.

Implementing this would essentially boil down to:

1. Adding an option to G_gui() to write the code to stdout rather than
calling popen("$GRASS_WISH").

2. Adding a global switch (e.g. --tcltk) to G_parser(), which makes
use of 1.

3. Copying the relevant portions of lib/gis/gui.tcl into tcltkgrass'
gui.tcl, with any necessary modifications.

4. Changing the execute procedure to e.g.:

  proc execute {cmd} {
      eval [exec -- $cmd --tcltk 2>@ stderr]
  }

So, execute would simply invoke the program to get the Tcl/Tk code for
the dialog, then evaluate it. The actual execution of the program
(with arguments) would occur when the user clicked the dialog's "Run"
button.

In short, while execute currently looks very much like spawn,
ultimately it could look very different.

I've committed the changes to implement the above. The actual changes
to tcltkgrass were trivial, essentially just adding:

  source $env(GISBASE)/etc/gui.tcl

and changing the execute procedure to:

  proc execute {cmd} {
      global dlg path
  
      set code [exec -- $cmd --tcltk]
  
      set path .dialog$dlg
      toplevel $path
      eval $code
  }

The changes to lib/gis/parser.c were also fairly straightforward; the
bulk of G_gui() has been moved into a separate function, generate_tcl.
G_gui() spawns wish, writes "source $env(GISBASE)/etc/gui.tcl" then
calls generate_tcl() with the pipe as the destination. A new function,
G_tcltk simply calls generate_tcl() with stdout as the destination.

Also, G_parser() now supports --ui (call G_gui()) and --tcltk (call
G_tcltk()) switches.

Most of the work was in making lib/gis/gui.tcl compatible with
tcltkgrass, primarily allowing multiple dialogs to coexist
simultaneously. One side effect of this is that tcltkgrass' memory
usage will grow with use. Each dialog adds new entries to the global
opt() array; at present, these entries aren't removed when the dialog
is closed.

While I can still remember how this all works, I'll provide a few
notes regarding the structure of lib/gis/gui.tcl:

1. The central data store is the opt() array. Keys into this array
always start with the dialog ID. Per-dialog keys consist of the dialog
ID and the name:

  opt($dlg,path) # prefix for widget paths
  opt($dlg,root) # top-level widget path
  opt($dlg,suf) # prefix for ScrollableFrame children
  opt($dlg,outtext) # path to the output widget
  opt($dlg,pgm_name) # program name
  opt($dlg,nopt) # number of options/flags

while per-field keys consist of the dialog ID, followed by the field
ID followed by the name:

  opt($dlg,$optn,answer) # string: opt->answer
  opt($dlg,$optn,class) # "multi", "opt" or "flag"
  opt($dlg,$optn,desc) # string: opt->description
  opt($dlg,$optn,multi) # boolean: opt->multiple
  opt($dlg,$optn,name) # string: opt->key or flag->key
  opt($dlg,$optn,nmulti) # integer: number of items in opt->options
  opt($dlg,$optn,options) # string: opt->options
  opt($dlg,$optn,prompt) # string: opt->gisprompt
  opt($dlg,$optn,required)# boolean: opt->required
  opt($dlg,$optn,type) # "integer", "float" or "string": opt->type
  opt($dlg,$optn,val) # contents of entry/combo widget (-textvariable)

The class and nmulti entries are computed within gui.tcl, while the
val entries are set by the widget via -textvariable. The rest come
directly from parser.c (i.e. from the optlist argument to add_option,
or from the key/desc arguments to add_flag).

Also, in the checkbox case (opt->multiple && opt->options != NULL),
there are per-option keys, consisting of the dialog ID, the field ID,
the name then the option ID:

  opt($dlg,$optn,val,$i) # checkbox state (-variable)
  opt($dlg,$optn,valname,$i) # corresponding item from opt->options

2. There are two other global variables: dlg and path. $dlg is the
current dialog ID (incremented for each call to begin_dialog), while
$path is the widget path prefix for children of the top-level widget.

For the G_gui() case, $path will always be null, so e.g. $path.run
will evaluate to just ".run". This value is stored in opt($dlg,path)
when the dialog is created, while opt($dlg,root) holds the actual path
to the top-level widget (if the path entry is null, the root entry
will be ".", otherwise the two will be identical).

For tcltkgrass, path is set to the path to the new top-level widget
which is created for that dialog prior to executing the code from
"<cmd> --tcltk".

3. lib/gis/parser.c uses the following four procedures from the bottom
of gui.tcl:

  proc begin_dialog {pgm desc}
  proc add_option {optn optlist} (for each option)
  proc add_flag {optn key desc} (for each flag)
  proc end_dialog {n}

In turn, these are the only procedures which reference the dlg and
path global variables; everything else has them passed as parameters.
The only global variables used by the other procedures are $opt() and
$env().

Any changes to this interface need to be synchronised between parser.c
and gui.tcl.

4. Ultimately, as additional features are added to tcltkgrass, some of
the code in gui.tcl will need to diverge into standalone (G_gui()) and
tcltkgrass versions. As that occurs, it would be a good idea to give
some thought to sharing as much code as is practical (rather than just
copying the whole of gui.tcl into tcltkgrass).

--
Glynn Clements <glynn.clements@virgin.net>

Also, G_parser() now supports --ui (call G_gui()) and --tcltk (call
G_tcltk()) switches.

any reason for restricting this to the first command line arg?

Hamish

Hamish wrote:

> Also, G_parser() now supports --ui (call G_gui()) and --tcltk (call
> G_tcltk()) switches.

any reason for restricting this to the first command line arg?

All of the global switches cause argument processing to be aborted, so
there isn't any point in passing any other arguments. If any of these
switches are given, they are invariably the only argument.

--
Glynn Clements <glynn.clements@virgin.net>