[GRASS-dev] grass and python

[William_Kyngesburye]

I've been working on an alternative Mac GRASS.app startup - currently it uses the OnMyCommand droplet as an app template, but I'm looking for a more open source, buildable kind of startup, and found py2app. (Lorenzo's AppleScript Studio method could work, but I don't want to have to deal with Xcode.)

I have something that works, but it got me thinking about the wxpython gui developments. I haven't been paying much attention to that and haven't had a chance to try it yet. So, py2app offers a cool possibility - a completely selfcontained Mac application, and its cousin py2exe might do the same for Windows.

My question is, does, or is it possible in the future for, wxpython grass to start directly from python? Or like the tcltk GUI, does it need the GRASS shell running first for the initial environment setup? It would be nice for the wxpython GUI to setup and maintain the environment that init.sh does. Then it could let the user open a Terminal or other shell (maybe a pseudo-terminal within python?) to run commands the old fashioned way if desired, but not require that. Also, it would keep a persistent 'GRASS.app' application in the Dock while 'GRASS' is running.

My current python startup app just fires up the normal init.sh startup and quits, leaving GRASS running in the Terminal and the GUI in TclTk X11, and disappears from the Dock. While the OnMyCommand method at least stays around in the Dock until GRASS is quit from the Terminal (even tho it doesn't do anything after starting GRASS, it's nice to have that visual clue in the Dock that GRASS is running).

Another question about the wxpython GUI - does or will it handle multiple GRASS sessions at once, kind of like having multiple documents (ie mapsets) open in an application? A Mac application normally can only open in one instance, unlike running different instances from multiple Terminal windows. I just realized that my current OMC app can only run one at a time, since the app stays open, yet the python method I'm working on can open multiple instances of GRASS, since the app itself quits right away, leaving GRASS running (a bit confusing for a Mac user, and especially so since there is no hard connection between the GUI and the Terminal shell that opened it).

One nice thing about the py2app setup is that it can locate and create local copies of all dependencies (python modules and dynamic libraries and frameworks) needed plus a minimal copy of the python runtime environment. Then the user doesn't have to worry about which Python they have installed, that might be too old or too new, or having all the needed modules installed. With the current init.sh startup, it would have a hard time locating dependencies for this since it doesn't really 'see' the internals of the wxpython grass gui, though it might since it seems to be pretty rigourous in identifying stuff like that.

-----
William Kyngesburye <kyngchaos@kyngchaos.com>
http://www.kyngchaos.com/

[Trillian] What are you supposed to do WITH a maniacally depressed robot?

[Marvin] You think you have problems? What are you supposed to do if you ARE a maniacally depressed robot? No, don't try and answer, I'm 50,000 times more intelligent than you and even I don't know the answer...

- HitchHiker's Guide to the Galaxy

[Trevor_Wiens2]

On Fri, 11 Aug 2006 11:19:48 -0500
William Kyngesburye <woklist@kyngchaos.com> wrote:

I've been working on an alternative Mac GRASS.app startup - currently
it uses the OnMyCommand droplet as an app template, but I'm looking
for a more open source, buildable kind of startup, and found py2app.
(Lorenzo's AppleScript Studio method could work, but I don't want to
have to deal with Xcode.)

I have something that works, but it got me thinking about the
wxpython gui developments. I haven't been paying much attention to
that and haven't had a chance to try it yet. So, py2app offers a
cool possibility - a completely selfcontained Mac application, and
its cousin py2exe might do the same for Windows.

My question is, does, or is it possible in the future for, wxpython
grass to start directly from python?

This is a good question and there has been some off list discussion
about this. What I've researched and am (very slowly implementing) is a
way to wrap the shell within the python GUI (optionally) so that users
will be able to use the shell of their choice with the new GUI.
Obviously a python shell would be default, but bash, etc would be
available. Together with this I'm planning on building a socket based
interface to the display tree that will be callable with a few simple
commands from whichever shell the user chooses. The advantage over the
existing display architecture is that it will be possible to manipulate
the display layers and redisplay from command line without having to
reissue each draw command. Another advantage is that the command line
use and the GUI are synchronized. One of the strengths of the new
gis.m is that you can have different views of the same data in
different windows but the difficulty for CLI based users is that
one can't EASILY manipulate those views from CLI. I'm trying to solve
this problem as well as make CLI use more elegant and GUI programming
simpler. At this point nothing is implemented as I'm in the research
and planning stage.

Or like the tcltk GUI, does it
need the GRASS shell running first for the initial environment
setup? It would be nice for the wxpython GUI to setup and maintain
the environment that init.sh does. Then it could let the user open a
Terminal or other shell (maybe a pseudo-terminal within python?) to
run commands the old fashioned way if desired, but not require that.
Also, it would keep a persistent 'GRASS.app' application in the Dock
while 'GRASS' is running.

My current python startup app just fires up the normal init.sh
startup and quits, leaving GRASS running in the Terminal and the GUI
in TclTk X11, and disappears from the Dock. While the OnMyCommand
method at least stays around in the Dock until GRASS is quit from the
Terminal (even tho it doesn't do anything after starting GRASS, it's
nice to have that visual clue in the Dock that GRASS is running).

Another question about the wxpython GUI - does or will it handle
multiple GRASS sessions at once, kind of like having multiple
documents (ie mapsets) open in an application?

From my point of view, this idea conflicts with GRASS architecture.

Still, if for Mac users running multiple instances was desirable but
problematic as the application wouldn't be shell dependent, it would
be possible to program the facility to open another copy of the GUI for
another mapset. Hmmm... Perhaps it might be useful to think of what
functionality this would achieve and then think if there is a more
elegant solution, such as gis layer (not in the convoluted vector
terminology sense) file browser which would enable one to look at
other mapsets, etc. Associated with such a GUI tool would obviously
come copying and reprojection, but since I'm not sure what you are
thinking of, this idea may be way off base.

A Mac application
normally can only open in one instance, unlike running different
instances from multiple Terminal windows. I just realized that my
current OMC app can only run one at a time, since the app stays open,
yet the python method I'm working on can open multiple instances of
GRASS, since the app itself quits right away, leaving GRASS running
(a bit confusing for a Mac user, and especially so since there is no
hard connection between the GUI and the Terminal shell that opened it).

One nice thing about the py2app setup is that it can locate and
create local copies of all dependencies (python modules and dynamic
libraries and frameworks) needed plus a minimal copy of the python
runtime environment. Then the user doesn't have to worry about which
Python they have installed, that might be too old or too new, or
having all the needed modules installed. With the current init.sh
startup, it would have a hard time locating dependencies for this
since it doesn't really 'see' the internals of the wxpython grass
gui, though it might since it seems to be pretty rigourous in
identifying stuff like that.

-----
William Kyngesburye <kyngchaos@kyngchaos.com>
http://www.kyngchaos.com/

T
--
Trevor Wiens
twiens@interbaun.com

The significant problems that we face cannot be solved at the same
level of thinking we were at when we created them.
(Albert Einstein)

[David_Finlayson]

It is quite possible to initialize and run grass without Bash. I have a simi-working python script that fires up GRASS using IPython as it’s shell. In fact, since Python abstracts os calls such as creating temp files, setting environment variables, etc, it is actually much easier to write a cross-platform initialization script in Python than in Bash. HOWEVER (and this is big), if you do not have a traditional unix shell running grass, none of the nearly 100 or so scripts that are distributed with GRASS—and that depend on bash—will work.

Many standard GRASS commands are really shell scripts. Every single one of them will need to be ported to your new shell if this isn’t compatible with Bash. I stopped working on this when I realized how disruptive this was going to be (and how unlikely this would be supported by the core GRASS community). I don’t think the vast majority of GRASS developers want to move to a Python environment (or Ruby or Perl or cmd.exe, etc). So, basically, this is an idea dead-on-arrival. It would amount to a fork of GRASS and new users would have a terrible time figuring out which shell they were supposed to be using for each script.

I think that GRASS is intimately tied to a Posix-compliant shell. There needs to be a standard macro-language for scripting and Bash seems as good as any.

David

On 8/11/06, Trevor Wiens <twiens@interbaun.com> wrote:

On Fri, 11 Aug 2006 11:19:48 -0500
William Kyngesburye <woklist@kyngchaos.com> wrote:

I’ve been working on an alternative Mac GRASS.app startup - currently
it uses the OnMyCommand droplet as an app template, but I’m looking
for a more open source, buildable kind of startup, and found py2app.
(Lorenzo’s AppleScript Studio method could work, but I don’t want to
have to deal with Xcode.)

I have something that works, but it got me thinking about the
wxpython gui developments. I haven’t been paying much attention to
that and haven’t had a chance to try it yet. So, py2app offers a
cool possibility - a completely selfcontained Mac application, and
its cousin py2exe might do the same for Windows.

My question is, does, or is it possible in the future for, wxpython
grass to start directly from python?

This is a good question and there has been some off list discussion
about this. What I’ve researched and am (very slowly implementing) is a
way to wrap the shell within the python GUI (optionally) so that users
will be able to use the shell of their choice with the new GUI.
Obviously a python shell would be default, but bash, etc would be
available. Together with this I’m planning on building a socket based
interface to the display tree that will be callable with a few simple
commands from whichever shell the user chooses. The advantage over the
existing display architecture is that it will be possible to manipulate
the display layers and redisplay from command line without having to
reissue each draw command. Another advantage is that the command line
use and the GUI are synchronized. One of the strengths of the new
gis.m is that you can have different views of the same data in
different windows but the difficulty for CLI based users is that
one can’t EASILY manipulate those views from CLI. I’m trying to solve
this problem as well as make CLI use more elegant and GUI programming
simpler. At this point nothing is implemented as I’m in the research
and planning stage.

Or like the tcltk GUI, does it
need the GRASS shell running first for the initial environment
setup? It would be nice for the wxpython GUI to setup and maintain
the environment that init.sh does. Then it could let the user open a
Terminal or other shell (maybe a pseudo-terminal within python?) to
run commands the old fashioned way if desired, but not require that.
Also, it would keep a persistent ’ GRASS.app’ application in the Dock
while ‘GRASS’ is running.

My current python startup app just fires up the normal init.sh
startup and quits, leaving GRASS running in the Terminal and the GUI
in TclTk X11, and disappears from the Dock. While the OnMyCommand
method at least stays around in the Dock until GRASS is quit from the
Terminal (even tho it doesn’t do anything after starting GRASS, it’s
nice to have that visual clue in the Dock that GRASS is running).

Another question about the wxpython GUI - does or will it handle
multiple GRASS sessions at once, kind of like having multiple
documents (ie mapsets) open in an application?

From my point of view, this idea conflicts with GRASS architecture.
Still, if for Mac users running multiple instances was desirable but
problematic as the application wouldn’t be shell dependent, it would
be possible to program the facility to open another copy of the GUI for
another mapset. Hmmm… Perhaps it might be useful to think of what
functionality this would achieve and then think if there is a more
elegant solution, such as gis layer (not in the convoluted vector
terminology sense) file browser which would enable one to look at
other mapsets, etc. Associated with such a GUI tool would obviously
come copying and reprojection, but since I’m not sure what you are
thinking of, this idea may be way off base.

A Mac application
normally can only open in one instance, unlike running different
instances from multiple Terminal windows. I just realized that my
current OMC app can only run one at a time, since the app stays open,
yet the python method I’m working on can open multiple instances of
GRASS, since the app itself quits right away, leaving GRASS running
(a bit confusing for a Mac user, and especially so since there is no
hard connection between the GUI and the Terminal shell that opened it).

One nice thing about the py2app setup is that it can locate and
create local copies of all dependencies (python modules and dynamic
libraries and frameworks) needed plus a minimal copy of the python
runtime environment. Then the user doesn’t have to worry about which
Python they have installed, that might be too old or too new, or
having all the needed modules installed. With the current init.sh
startup, it would have a hard time locating dependencies for this
since it doesn’t really ‘see’ the internals of the wxpython grass
gui, though it might since it seems to be pretty rigourous in
identifying stuff like that.

---

William Kyngesburye < kyngchaos@kyngchaos.com>
http://www.kyngchaos.com/

T

Trevor Wiens
twiens@interbaun.com

The significant problems that we face cannot be solved at the same
level of thinking we were at when we created them.
(Albert Einstein)

---

grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

–
David Finlayson

[Glynn_Clements1]

David Finlayson wrote:

It is quite possible to initialize and run grass without Bash. I have a
simi-working python script that fires up GRASS using IPython as it's shell.
In fact, since Python abstracts os calls such as creating temp files,
setting environment variables, etc, it is actually much easier to write a
cross-platform initialization script in Python than in Bash. HOWEVER (and
this is big), if you do not have a traditional unix shell running grass,
none of the nearly 100 or so scripts that are distributed with GRASS---and
that depend on bash---will work.

Many standard GRASS commands are really shell scripts. Every single one of
them will need to be ported to your new shell if this isn't compatible with
Bash. I stopped working on this when I realized how disruptive this was
going to be (and how unlikely this would be supported by the core GRASS
community). I don't think the vast majority of GRASS developers want to move
to a Python environment (or Ruby or Perl or cmd.exe, etc). So, basically,
this is an idea dead-on-arrival. It would amount to a fork of GRASS and new
users would have a terrible time figuring out which shell they were supposed
to be using for each script.

I think that GRASS is intimately tied to a Posix-compliant shell. There
needs to be a standard macro-language for scripting and Bash seems as good
as any.

The one thing that Bourne shell (not necessarily bash) has going for
it is its install base. It exists on every Unix-like system, and on
non-Unix systems, it's as widespread as any other portable interpreted
language.

Personally, I'd be more than happy to drop Bourne shell in favour of
some other language, *if* there was some other language which was
sufficiently trivial to install and use on systems which didn't
already have it.

The biggest problem is Windows, specifically the fact that it doesn't
support scripts which specify their interpreter in the file itself
(i.e. the #! mechanism on Unix); the interpreter has to be specified
through the file's extension.

This isn't insurmountable. We could force the appropriate extensions
onto script files for Windows installations, and you can use the
PATHEXT variable to eliminate the need to type the extension.

However, I don't know whether this works through system()-type
functions in other languages, or whether you would have to include the
script's extension there. Requiring scripts (or compiled programs) to
conditionally include the extension only on Windows would be a
nuisance, and unconditionally adding an extension would be a nuisance
on Unix systems.

If anyone can find a real language which is "good enough" on all of
Unix, Windows (Cygwin and native) and OSX, I'll quite happily rewrite
all of GRASS' shell scripts.

Apart from the fact that Bourne shell basically sucks in every way
imaginable as a programming languague, it's further hampered by the
fact that all of the common Windows ports try to provide some degree
of Unix compatibility (e.g. virtual Unix filesystem) which ultimately
ends up causing problems.

--
Glynn Clements <glynn@gclements.plus.com>

[William_Kyngesburye]

On Aug 11, 2006, at 10:55 PM, Trevor Wiens wrote:

My question is, does, or is it possible in the future for, wxpython
grass to start directly from python?

This is a good question and there has been some off list discussion
about this. What I've researched and am (very slowly implementing) is a
way to wrap the shell within the python GUI (optionally) so that users
will be able to use the shell of their choice with the new GUI.

...

simpler. At this point nothing is implemented as I'm in the research
and planning stage.

Cool. I'll wait to see what you come up with.

Another question about the wxpython GUI - does or will it handle
multiple GRASS sessions at once, kind of like having multiple
documents (ie mapsets) open in an application?

From my point of view, this idea conflicts with GRASS architecture.

Right, understood...

Still, if for Mac users running multiple instances was desirable but
problematic as the application wouldn't be shell dependent, it would
be possible to program the facility to open another copy of the GUI for
another mapset. Hmmm... Perhaps it might be useful to think of what
functionality this would achieve and then think if there is a more
elegant solution, such as gis layer (not in the convoluted vector
terminology sense) file browser which would enable one to look at
other mapsets, etc. Associated with such a GUI tool would obviously
come copying and reprojection, but since I'm not sure what you are
thinking of, this idea may be way off base.

... But, yes, a Mac application normally only can run in one instance. So the application would have to be able to handle working with multiple working mapsets. We'd have to figure out what a 'document' would be, like a working environment, connected to a single mapset, which can have one or more displays/views, command windows, a log window, like the GUI is now (can it have more than one display?). Keeping them connected visually so the user knows which window belongs to which environment might be tricky (it already is when there is more than one GUI running).

Maybe a single-document workspace idea might be better - have a single window with panes for the various parts, kinda like the NVIZ window. Each window object can easily keep track of its own environment.

-----
William Kyngesburye <kyngchaos@kyngchaos.com>
http://www.kyngchaos.com/

Earth: "Mostly harmless"

- revised entry in the HitchHiker's Guide to the Galaxy

[William_Kyngesburye]

On Aug 12, 2006, at 12:38 AM, David Finlayson wrote:

It is quite possible to initialize and run grass without Bash. I have a simi-working python script that fires up GRASS using IPython as it's shell. In fact, since Python abstracts os calls such as creating temp files, setting environment variables, etc, it is actually much easier to write a cross-platform initialization script in Python than in Bash. HOWEVER (and this is big), if you do not have a traditional unix shell running grass, none of the nearly 100 or so scripts that are distributed with GRASS---and that depend on bash---will work.

But, as long as the Python app sets up and maintains that shell environment, shouldn't the shell scripts work, if run from the Python environment? something like (pardon, just getting started with python):

os.environ to set up environment (probably what your experimental python startup does)

os.popen to run the shell script - uses os.environ setup, or is that where it needs a running shell? It's possible to start it in a shell, like a new Terminal window on OS X, tho a bit messy. Maybe there's a python-driven shell interpreter?

Many standard GRASS commands are really shell scripts. Every single one of them will need to be ported to your new shell if this isn't compatible with Bash. I stopped working on this when I realized how disruptive this was going to be (and how unlikely this would be supported by the core GRASS community). I don't think the vast majority of GRASS developers want to move to a Python environment (or Ruby or Perl or cmd.exe, etc). So, basically, this is an idea dead-on-arrival. It would amount to a fork of GRASS and new users would have a terrible time figuring out which shell they were supposed to be using for each script.

I think that GRASS is intimately tied to a Posix-compliant shell. There needs to be a standard macro-language for scripting and Bash seems as good as any.

Coming from a more object-oriented programming perspective, I find unix shell scripting to be a nightmare.

If the wxpython GUI takes off as the main GRASS GUI, there will probably be less resistance to moving to Python for all or most GRASS scripting.

-----
William Kyngesburye <kyngchaos@kyngchaos.com>
http://www.kyngchaos.com/

"This is a question about the past, is it? ... How can I tell that the past isn't a fiction designed to account for the discrepancy between my immediate physical sensations and my state of mind?"

- The Ruler of the Universe

[cmbarton]

William,

Glynn et al have already answered some of this. I'll add a few more
comments.

I've been through some of this recently working with the agent modeling
group in my research team.

GRASS doesn't run (in the sense of running a process). That is, when you run
init.sh, it doesn't really do anything except set up a number of
environmental variables to let you run (in the sense of starting a process)
GRASS modules.

In this respect, you can "start" GRASS in pretty near anything. You need
some kind of shell to process the commands, but that's about it. We're now
running GRASS commands from within a DEVS JAVA agent modeling environment.
(This is very cool by the way)

So yes, you can run GRASS from within Python. The current wxPython prototype
takes a somewhat different approach and sends GRASS commands to the default
shell (whatever it is).

Glynn's and Trevor's comments about GRASS and the BASH shell are apropos.
Currently it is a good common denominator on most systems with GRASS today,
but difficult from a programming perspective and problematic for Widows.

But what can we replace it with? Right now there is no good answer. If the
wxPython UI takes off, we might be able to revisit Python as a default shell
in the future, but I don't think we're there yet.

In order to support multiple sessions, I think you simply need to start
multiple shells (i.e., command parser with environment settings) for GRASS
commands to run in. Lorenzo's binaries will do that now. The biggest
convenience for me of multiple GRASS sessions is being able to work in
multiple locations simultaneously (though only for brief times or I get
confused).

I've only messed with the app building a couple of times--and didn't get it
right yet.

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: William Kyngesburye <woklist@kyngchaos.com>
Reply-To: William Kyngesburye <kyngchaos@kyngchaos.com>
Date: Fri, 11 Aug 2006 11:19:48 -0500
To: Michael Barton <michael.barton@asu.edu>
Cc: <grass-dev@grass.itc.it>
Subject: [GRASS-dev] grass and python

I've been working on an alternative Mac GRASS.app startup - currently
it uses the OnMyCommand droplet as an app template, but I'm looking
for a more open source, buildable kind of startup, and found py2app.
(Lorenzo's AppleScript Studio method could work, but I don't want to
have to deal with Xcode.)

I have something that works, but it got me thinking about the
wxpython gui developments. I haven't been paying much attention to
that and haven't had a chance to try it yet. So, py2app offers a
cool possibility - a completely selfcontained Mac application, and
its cousin py2exe might do the same for Windows.

My question is, does, or is it possible in the future for, wxpython
grass to start directly from python? Or like the tcltk GUI, does it
need the GRASS shell running first for the initial environment
setup? It would be nice for the wxpython GUI to setup and maintain
the environment that init.sh does. Then it could let the user open a
Terminal or other shell (maybe a pseudo-terminal within python?) to
run commands the old fashioned way if desired, but not require that.
Also, it would keep a persistent 'GRASS.app' application in the Dock
while 'GRASS' is running.

My current python startup app just fires up the normal init.sh
startup and quits, leaving GRASS running in the Terminal and the GUI
in TclTk X11, and disappears from the Dock. While the OnMyCommand
method at least stays around in the Dock until GRASS is quit from the
Terminal (even tho it doesn't do anything after starting GRASS, it's
nice to have that visual clue in the Dock that GRASS is running).

Another question about the wxpython GUI - does or will it handle
multiple GRASS sessions at once, kind of like having multiple
documents (ie mapsets) open in an application? A Mac application
normally can only open in one instance, unlike running different
instances from multiple Terminal windows. I just realized that my
current OMC app can only run one at a time, since the app stays open,
yet the python method I'm working on can open multiple instances of
GRASS, since the app itself quits right away, leaving GRASS running
(a bit confusing for a Mac user, and especially so since there is no
hard connection between the GUI and the Terminal shell that opened it).

One nice thing about the py2app setup is that it can locate and
create local copies of all dependencies (python modules and dynamic
libraries and frameworks) needed plus a minimal copy of the python
runtime environment. Then the user doesn't have to worry about which
Python they have installed, that might be too old or too new, or
having all the needed modules installed. With the current init.sh
startup, it would have a hard time locating dependencies for this
since it doesn't really 'see' the internals of the wxpython grass
gui, though it might since it seems to be pretty rigourous in
identifying stuff like that.

-----
William Kyngesburye <kyngchaos@kyngchaos.com>
http://www.kyngchaos.com/

[Trillian] What are you supposed to do WITH a maniacally depressed
robot?

[Marvin] You think you have problems? What are you supposed to do
if you ARE a maniacally depressed robot? No, don't try and answer,
I'm 50,000 times more intelligent than you and even I don't know the
answer...

- HitchHiker's Guide to the Galaxy

[cmbarton]

Trevor,

The current prototype at least partly solves these issues.

You can now issue multiple d.* commands, separate by commas on the CLI.
These are parsed into a list of d.* commands and processed in a loop.

Try d.rast elevation_dem, d.vect streams color=blue

Also, any d.* commands are now processed for display into whichever map
display has the focus. Try opening a second display and typing some
different d.* commmands; click on the first display and type some new d.*
commands.

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: Trevor Wiens <twiens@interbaun.com>
Date: Fri, 11 Aug 2006 21:55:47 -0600
To: William Kyngesburye <kyngchaos@kyngchaos.com>
Cc: <woklist@kyngchaos.com>, Michael Barton <michael.barton@asu.edu>,
<grass-dev@grass.itc.it>
Subject: Re: [GRASS-dev] grass and python

This is a good question and there has been some off list discussion
about this. What I've researched and am (very slowly implementing) is a
way to wrap the shell within the python GUI (optionally) so that users
will be able to use the shell of their choice with the new GUI.
Obviously a python shell would be default, but bash, etc would be
available. Together with this I'm planning on building a socket based
interface to the display tree that will be callable with a few simple
commands from whichever shell the user chooses. The advantage over the
existing display architecture is that it will be possible to manipulate
the display layers and redisplay from command line without having to
reissue each draw command. Another advantage is that the command line
use and the GUI are synchronized. One of the strengths of the new
gis.m is that you can have different views of the same data in
different windows but the difficulty for CLI based users is that
one can't EASILY manipulate those views from CLI. I'm trying to solve
this problem as well as make CLI use more elegant and GUI programming
simpler. At this point nothing is implemented as I'm in the research
and planning stage.

[cmbarton]

William,

See additional responses below.

Michal
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: William Kyngesburye <woklist@kyngchaos.com>
Reply-To: William Kyngesburye <kyngchaos@kyngchaos.com>
Date: Sat, 12 Aug 2006 10:15:15 -0500
To: David Finlayson <david.p.finlayson@gmail.com>
Cc: <grass-dev@grass.itc.it>
Subject: Re: [GRASS-dev] grass and python

But, as long as the Python app sets up and maintains that shell
environment, shouldn't the shell scripts work, if run from the Python
environment? something like (pardon, just getting started with python):

Bash scripts will work if it sets up a Bash environment, but not necessarily
if it sets up another shell environment. The command prompt in Unix is
actually a running shell program that provides an interface for shell
commands. There are multiple shells possible, but Bash has become a common
denominator for GRASS and many other programs.

os.environ to set up environment (probably what your experimental
python startup does)

It use os.environ to read currently set environmental variables. There is
some cautioning in the Python manual about using it to set environmental
variables on different platforms.

os.popen to run the shell script - uses os.environ setup, or is that
where it needs a running shell? It's possible to start it in a
shell, like a new Terminal window on OS X, tho a bit messy. Maybe
there's a python-driven shell interpreter?

The Python cmd module may do this, but I haven't yet experimented with it.
Maybe David knows.

Many standard GRASS commands are really shell scripts. Every single
one of them will need to be ported to your new shell if this isn't
compatible with Bash. I stopped working on this when I realized how
disruptive this was going to be (and how unlikely this would be
supported by the core GRASS community). I don't think the vast
majority of GRASS developers want to move to a Python environment
(or Ruby or Perl or cmd.exe, etc). So, basically, this is an idea
dead-on-arrival. It would amount to a fork of GRASS and new users
would have a terrible time figuring out which shell they were
supposed to be using for each script.

I think that GRASS is intimately tied to a Posix-compliant shell.
There needs to be a standard macro-language for scripting and Bash
seems as good as any.

Coming from a more object-oriented programming perspective, I find
unix shell scripting to be a nightmare.

If the wxpython GUI takes off as the main GRASS GUI, there will
probably be less resistance to moving to Python for all or most GRASS
scripting.

We'll see. I've done my share of shell scripting and do like what I'm
learning about Python. I'm especially encouraged that so many other people
know something about it too.

Michael

[William_Kyngesburye]

On Aug 12, 2006, at 12:45 PM, Michael Barton wrote:

In order to support multiple sessions, I think you simply need to start
multiple shells (i.e., command parser with environment settings) for GRASS
commands to run in. Lorenzo's binaries will do that now. The biggest
convenience for me of multiple GRASS sessions is being able to work in
multiple locations simultaneously (though only for brief times or I get
confused).

I sometimes also run multiple GRASS sessions (easy - new Terminal window) - handy if I run something in one mapset that takes a LONG time, I can putter around with stuff in another mapset. I think Lorenzo's handling multiple sessions is because his OSX app actually does manage multiple sessions (I don't know), much like I'm thinking about how a wxpython might manage multiple sessions, or workspaces, the 'document' analogy. Or maybe it just quits after starting GRASS.

My OnMyCommand startup just passes the startup on to the Terminal and the grass init.sh, but OMC app happens to continue running (doing nothing) while the GRASS shell continues to run. Thus, the OMC app can't run again to start another GRASS session.

The Python py2app startup (independent of the wxpython GUI) I came up with can run multiple sessions, since the python startup app starts the GRASS shell in the Terminal, then quits, leaving GRASS running and leaving the user the possibility of running the python startup again.

My OMC and python startups don't manage GRASS sessions.

hmmm, at least I should try to get the OMC startup to not wait around for GRASS to quit for now.

I've only messed with the app building a couple of times--and didn't get it
right yet.

GRASS.app building, or Mac OS X app building in general? Xcode and Mac app development is much too hardcore for me. I'm better off with casual programming and scripting and tinkering.

I have a working Python app startup as an alternative to the OMC method I currently use in my app binary. I want something buildable and distibutable as source that can either be easily used by anyone to build a GRASS.app from source or possibly even included in the GRASS source as an option. Then others could more easily play around with Mac app builds and customize them.

-----
William Kyngesburye <kyngchaos@kyngchaos.com>
http://www.kyngchaos.com/

Earth: "Mostly harmless"

- revised entry in the HitchHiker's Guide to the Galaxy

[cmbarton]

William,

I hope to get the wxPython stuff into the cvs early next week. This will
make it easier for you and others to work with it.

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: William Kyngesburye <woklist@kyngchaos.com>
Reply-To: William Kyngesburye <kyngchaos@kyngchaos.com>
Date: Sat, 12 Aug 2006 14:07:28 -0500
To: Michael Barton <michael.barton@asu.edu>
Cc: <grass-dev@grass.itc.it>
Subject: Re: [GRASS-dev] grass and python

On Aug 12, 2006, at 12:45 PM, Michael Barton wrote:

In order to support multiple sessions, I think you simply need to
start
multiple shells (i.e., command parser with environment settings)
for GRASS
commands to run in. Lorenzo's binaries will do that now. The biggest
convenience for me of multiple GRASS sessions is being able to work in
multiple locations simultaneously (though only for brief times or I
get
confused).

I sometimes also run multiple GRASS sessions (easy - new Terminal
window) - handy if I run something in one mapset that takes a LONG
time, I can putter around with stuff in another mapset. I think
Lorenzo's handling multiple sessions is because his OSX app actually
does manage multiple sessions (I don't know), much like I'm thinking
about how a wxpython might manage multiple sessions, or workspaces,
the 'document' analogy. Or maybe it just quits after starting GRASS.

My OnMyCommand startup just passes the startup on to the Terminal and
the grass init.sh, but OMC app happens to continue running (doing
nothing) while the GRASS shell continues to run. Thus, the OMC app
can't run again to start another GRASS session.

The Python py2app startup (independent of the wxpython GUI) I came up
with can run multiple sessions, since the python startup app starts
the GRASS shell in the Terminal, then quits, leaving GRASS running
and leaving the user the possibility of running the python startup
again.

My OMC and python startups don't manage GRASS sessions.

hmmm, at least I should try to get the OMC startup to not wait around
for GRASS to quit for now.

I've only messed with the app building a couple of times--and
didn't get it
right yet.

GRASS.app building, or Mac OS X app building in general? Xcode and
Mac app development is much too hardcore for me. I'm better off with
casual programming and scripting and tinkering.

I have a working Python app startup as an alternative to the OMC
method I currently use in my app binary. I want something buildable
and distibutable as source that can either be easily used by anyone
to build a GRASS.app from source or possibly even included in the
GRASS source as an option. Then others could more easily play around
with Mac app builds and customize them.

-----
William Kyngesburye <kyngchaos@kyngchaos.com>
http://www.kyngchaos.com/

Earth: "Mostly harmless"

- revised entry in the HitchHiker's Guide to the Galaxy

[Glynn_Clements1]

William Kyngesburye wrote:

> It is quite possible to initialize and run grass without Bash. I
> have a simi-working python script that fires up GRASS using IPython
> as it's shell. In fact, since Python abstracts os calls such as
> creating temp files, setting environment variables, etc, it is
> actually much easier to write a cross-platform initialization
> script in Python than in Bash. HOWEVER (and this is big), if you do
> not have a traditional unix shell running grass, none of the nearly
> 100 or so scripts that are distributed with GRASS---and that depend
> on bash---will work.

But, as long as the Python app sets up and maintains that shell
environment, shouldn't the shell scripts work, if run from the Python
environment? something like (pardon, just getting started with python):

I think that he was suggesting eliminating the need for a Bourne shell
(e.g. bash) altogether. Running Bourne-shell scripts from Python is
entirely feasible, provided that you actually have a Bourne shell
interpreter to run them with.

> Many standard GRASS commands are really shell scripts. Every single
> one of them will need to be ported to your new shell if this isn't
> compatible with Bash. I stopped working on this when I realized how
> disruptive this was going to be (and how unlikely this would be
> supported by the core GRASS community). I don't think the vast
> majority of GRASS developers want to move to a Python environment
> (or Ruby or Perl or cmd.exe, etc). So, basically, this is an idea
> dead-on-arrival. It would amount to a fork of GRASS and new users
> would have a terrible time figuring out which shell they were
> supposed to be using for each script.
>
> I think that GRASS is intimately tied to a Posix-compliant shell.
> There needs to be a standard macro-language for scripting and Bash
> seems as good as any.

Coming from a more object-oriented programming perspective, I find
unix shell scripting to be a nightmare.

If the wxpython GUI takes off as the main GRASS GUI, there will
probably be less resistance to moving to Python for all or most GRASS
scripting.

The main issue there is that scripting and interactive command-line
use are very different. Shells (Bourne shell, csh) were designed
primarily for interactive use, and tend to suck as programming
languages. OTOH, Python (and Perl etc) are decent programming
languages but aren't necessarily suitable as interactive shells.

Re-writing scripts in e.g. Python is fine, so long as it can be done
in such a way that the scripts can be run from an interactive shell.
This is where Windows becomes problematic, due to its lack of support
for extension-less scripts. Windows ports of bash usually get around
this by having the shell itself identify scripts and process the #!
syntax.

--
Glynn Clements <glynn@gclements.plus.com>

[Trevor_Wiens2]

On Sat, 12 Aug 2006 08:02:21 +0100
Glynn Clements <glynn@gclements.plus.com> wrote:

If anyone can find a real language which is "good enough" on all of
Unix, Windows (Cygwin and native) and OSX, I'll quite happily rewrite
all of GRASS' shell scripts.

Python is easy to install on Windows. So with an offer like that, I
can't help asking if any of the interactive Python shells are to your
knowledge good enough. Or perhaps what in you mind defines an
interactive shell environment that is good enough to replace bash?

Apart from the fact that Bourne shell basically sucks in every way
imaginable as a programming languague, it's further hampered by the
fact that all of the common Windows ports try to provide some degree
of Unix compatibility (e.g. virtual Unix filesystem) which ultimately
ends up causing problems.

--
Glynn Clements <glynn@gclements.plus.com>

T
--
Trevor Wiens
twiens@interbaun.com

The significant problems that we face cannot be solved at the same
level of thinking we were at when we created them.
(Albert Einstein)

[Glynn_Clements1]

Trevor Wiens wrote:

> If anyone can find a real language which is "good enough" on all of
> Unix, Windows (Cygwin and native) and OSX, I'll quite happily rewrite
> all of GRASS' shell scripts.

Python is easy to install on Windows. So with an offer like that, I
can't help asking if any of the interactive Python shells are to your
knowledge good enough. Or perhaps what in you mind defines an
interactive shell environment that is good enough to replace bash?

The main issue is the ability to have scripts which can be run as e.g.
"d.save", and not having to use "d.save.py". This needs to work both
from a shell and from system/exec/popen/spawn/etc functions provided
by various languages.

--
Glynn Clements <glynn@gclements.plus.com>

[Trevor_Wiens2]

On Sun, 13 Aug 2006 05:59:20 +0100
Glynn Clements <glynn@gclements.plus.com> wrote:

Trevor Wiens wrote:

> > If anyone can find a real language which is "good enough" on all of
> > Unix, Windows (Cygwin and native) and OSX, I'll quite happily rewrite
> > all of GRASS' shell scripts.
>
> Python is easy to install on Windows. So with an offer like that, I
> can't help asking if any of the interactive Python shells are to your
> knowledge good enough. Or perhaps what in you mind defines an
> interactive shell environment that is good enough to replace bash?

The main issue is the ability to have scripts which can be run as e.g.
"d.save", and not having to use "d.save.py". This needs to work both
from a shell and from system/exec/popen/spawn/etc functions provided
by various languages.

A series so simple wrapper scripts might be needed to make sure that
issuing a map.draw command executes map.draw.py, but it seems desirable
to me to make sure that CLI use of commands would be the same no matter
what the shell of choice.

T
--
Trevor Wiens
twiens@interbaun.com

The significant problems that we face cannot be solved at the same
level of thinking we were at when we created them.
(Albert Einstein)

[David_Finlayson]

The Python interpreter is interactive, but it is not a general purpose shell. You would need to wrap external programs in an os.system or os.popen call. One attractive way of getting around this is to pre-wrap each program in a python object. For example:

class Display:

def rast(self, rastername):

On 8/12/06, Glynn Clements <glynn@gclements.plus.com > wrote:

Trevor Wiens wrote:

If anyone can find a real language which is “good enough” on all of
Unix, Windows (Cygwin and native) and OSX, I’ll quite happily rewrite
all of GRASS’ shell scripts.

Python is easy to install on Windows. So with an offer like that, I
can’t help asking if any of the interactive Python shells are to your
knowledge good enough. Or perhaps what in you mind defines an
interactive shell environment that is good enough to replace bash?

The main issue is the ability to have scripts which can be run as e.g .
“d.save”, and not having to use “d.save.py”. This needs to work both
from a shell and from system/exec/popen/spawn/etc functions provided
by various languages.

–
Glynn Clements <glynn@gclements.plus.com>

–
David Finlayson

[David_Finlayson]

The Python interpreter is interactive, but it is not a general purpose shell. You would need to wrap external programs in an os.system or os.popen call. One attractive way of getting around this is to pre-wrap each program in a python object. For example:

class Display:

def rast(self, rastername):
os.system(“d.rast %s” % rastername)

Then the syntax in the Python interpreter would be:

import Display
d = Display.display ()
d.rast(rastername)

Which isn’t so bad. This is what things would look like in PyShell (or PyCrust) which is a wxWindows-compatible python interpreter that you can use in wxWindows GUI’s.

The most advanced Python interpreter is IPython. This has been tweaked for interactive use by the Scipy (numeric programming) folks so that some of the Python decorations can be dropped in the above example:

import Display
d = Display.display()
d.rast rastername

No more parenthesis needed. Also, IPython has “magic” operators that allow you to “shell-out” to the underlying shell. Simply prepend a command with a % and it will bypass the python interpreter and send it to the shell:

%ls

or

%dir

The IPython interpreter is designed for interactive use. It is the Scipy answer to Matlab’s interactive prompt. It has a lot of nice shell-like features and can be extended for use by GRASS (I think). Unfortunately, I don’t think IPython is as well integrated with Windows as Unix. At least on my casual observations.

David

On 8/13/06, David Finlayson <david.p.finlayson@gmail.com> wrote:

The Python interpreter is interactive, but it is not a general purpose shell. You would need to wrap external programs in an os.system or os.popen call. One attractive way of getting around this is to pre-wrap each program in a python object. For example:

class Display:

def rast(self, rastername):

On 8/12/06, Glynn Clements < glynn@gclements.plus.com > wrote:

Trevor Wiens wrote:

If anyone can find a real language which is “good enough” on all of
Unix, Windows (Cygwin and native) and OSX, I’ll quite happily rewrite
all of GRASS’ shell scripts.

Python is easy to install on Windows. So with an offer like that, I
can’t help asking if any of the interactive Python shells are to your
knowledge good enough. Or perhaps what in you mind defines an
interactive shell environment that is good enough to replace bash?

The main issue is the ability to have scripts which can be run as e.g .
“d.save”, and not having to use “d.save.py”. This needs to work both
from a shell and from system/exec/popen/spawn/etc functions provided
by various languages.

–
Glynn Clements <glynn@gclements.plus.com>

–

David Finlayson

–
David Finlayson

[cmbarton]

Look at the Python cmd module for this.

Michael

---

Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

---

From: David Finlayson david.p.finlayson@gmail.com
Date: Sun, 13 Aug 2006 21:39:33 -0700
To: Glynn Clements glynn@gclements.plus.com
Cc: Trevor Wiens twiens@interbaun.com, Michael Barton michael.barton@asu.edu, grass-dev@grass.itc.it, William@grass.itc.it, Kyngesburye kyngchaos@kyngchaos.com, woklist@kyngchaos.com
Subject: Re: [GRASS-dev] grass and python

The Python interpreter is interactive, but it is not a general purpose shell. You would need to wrap external programs in an os.system or os.popen call. One attractive way of getting around this is to pre-wrap each program in a python object. For example:

class Display:

def rast(self, rastername):

On 8/12/06, Glynn Clements <glynn@gclements.plus.com mailto:glynn@gclements.plus.com > wrote:

Trevor Wiens wrote:

If anyone can find a real language which is “good enough” on all of
Unix, Windows (Cygwin and native) and OSX, I’ll quite happily rewrite
all of GRASS’ shell scripts.

Python is easy to install on Windows. So with an offer like that, I
can’t help asking if any of the interactive Python shells are to your
knowledge good enough. Or perhaps what in you mind defines an
interactive shell environment that is good enough to replace bash?

The main issue is the ability to have scripts which can be run as e.g .
“d.save”, and not having to use "d.save.py http://d.save.py ". This needs to work both
from a shell and from system/exec/popen/spawn/etc functions provided
by various languages.

–
Glynn Clements glynn@gclements.plus.com

–
David Finlayson

[cmbarton]

I just put up new files (mapdisc.py, gism.py, and render.py) on my website.

I rewrote the display as a double buffered dc drawing. I HOPE this solves the display issues with Linux. Thuban seems to use a somewhat more primitive version of this, so I’m keeping my fingers crossed. Please let me know. If it works, I’ll package it all up for the last time and note it on the WIKI. Then I hope to move it to the cvs so all you can work on it.

I also added a couple new bells and whistles. By redoing the display to use a bufferedDC, it is easy to make graphic file output. Using a simply example from the wxPython cookbook, I added output to PNG files. It is easy to add other formats.

With regards to the discussion about postscript, wxPython has a wx.PrinterDC and wx.PostScriptDC, that will output to postscript.

I also have set it so that you can specify the pen for mouse drawing in the event handler for a display control button. As example, zooming in and out have different colored zoom-rectangles.

Currently, the little command line shell allow you to chain display commands. It simply puts them in a list and processes them in the order of the chain. You could make something more sophisticated like having add [command] and delete [command] to add them to a list, but this soon gets complicated (e.g., rearranging display commands in the list to control how they are rendered may be simpler by just cutting and pasting them in a chain rather than trying to remember where they are and doing a move or insert). You also have to watch out for possible conflicts with ‘real’ shell commands.

It’s still not clear to me what is gained by starting in a Bash shell, starting the python app, and have it listen for special commands coming from the shell that it can recognize and parse. Why not simply start in a python shell that can send any shell commands to the shell (whatever shell you want; this too can be set by a command or a default from a config file) and trap the 350 or so GRASS commands? Same effect and easier to code and control. Maybe I’m missing something.

Michael

---

Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

---

From: David Finlayson david.p.finlayson@gmail.com
Date: Sun, 13 Aug 2006 21:47:16 -0700
To: Glynn Clements glynn@gclements.plus.com
Cc: Trevor Wiens twiens@interbaun.com, Michael Barton michael.barton@asu.edu, grass-dev@grass.itc.it, William@grass.itc.it, Kyngesburye kyngchaos@kyngchaos.com, woklist@kyngchaos.com
Subject: Re: [GRASS-dev] grass and python

The Python interpreter is interactive, but it is not a general purpose shell. You would need to wrap external programs in an os.system or os.popen call. One attractive way of getting around this is to pre-wrap each program in a python object. For example:

class Display:

def rast(self, rastername):
os.system(“d.rast %s” % rastername)

Then the syntax in the Python interpreter would be:

import Display
d = Display.display ()
d.rast(rastername)

Which isn’t so bad. This is what things would look like in PyShell (or PyCrust) which is a wxWindows-compatible python interpreter that you can use in wxWindows GUI’s.

The most advanced Python interpreter is IPython. This has been tweaked for interactive use by the Scipy (numeric programming) folks so that some of the Python decorations can be dropped in the above example:

import Display
d = Display.display()
d.rast rastername

No more parenthesis needed. Also, IPython has “magic” operators that allow you to “shell-out” to the underlying shell. Simply prepend a command with a % and it will bypass the python interpreter and send it to the shell:

%ls

or

%dir

The IPython interpreter is designed for interactive use. It is the Scipy answer to Matlab’s interactive prompt. It has a lot of nice shell-like features and can be extended for use by GRASS (I think). Unfortunately, I don’t think IPython is as well integrated with Windows as Unix. At least on my casual observations.

David

On 8/13/06, David Finlayson david.p.finlayson@gmail.com wrote:

The Python interpreter is interactive, but it is not a general purpose shell. You would need to wrap external programs in an os.system or os.popen call. One attractive way of getting around this is to pre-wrap each program in a python object. For example:

class Display:

def rast(self, rastername):

On 8/12/06, Glynn Clements < glynn@gclements.plus.com mailto:glynn@gclements.plus.com > wrote:

Trevor Wiens wrote:

If anyone can find a real language which is “good enough” on all of
Unix, Windows (Cygwin and native) and OSX, I’ll quite happily rewrite
all of GRASS’ shell scripts.

Python is easy to install on Windows. So with an offer like that, I
can’t help asking if any of the interactive Python shells are to your
knowledge good enough. Or perhaps what in you mind defines an
interactive shell environment that is good enough to replace bash?

The main issue is the ability to have scripts which can be run as e.g .
“d.save”, and not having to use "d.save.py http://d.save.py ". This needs to work both
from a shell and from system/exec/popen/spawn/etc functions provided
by various languages.

–
Glynn Clements glynn@gclements.plus.com

–
David Finlayson

–
David Finlayson

[Glynn_Clements1]

David Finlayson wrote:

The Python interpreter is interactive, but it is not a general purpose
shell. You would need to wrap external programs in an os.system or
os.popencall. One attractive way of getting around this is to pre-wrap
each program
in a python object. For example:

class Display:

    def rast(self, rastername):
        os.system("d.rast %s" % rastername)

What does os.system() do on Windows? If it passes the command directly
to the underlying OS function (e.g. CreateProcess()), this will fail
when applied to a script, as Windows won't figure out that it needs to
run the script via an interpreter.

On Unix, if you call execve() (directly or indirectly) on a script
(any executable file beginning with "#!"), the kernel will read the
name of the interpreter from the #! line and execute it, passing the
path to the script as an argument.

AFAIK, Windows doesn't have any such functionality; it relies upon
extensions to determine what to do with a file. Windows ports of bash
handle scripts by inspecting the "executable" themselves.

--
Glynn Clements <glynn@gclements.plus.com>