[GRASS-dev] A portible shell for GRASS 7+ ?

[David_Finlayson]

IPython is an advanced interactive shell for Python. It has a lot of
features that make interactive use of Python very powerful.

I think that we could write some wrappers around the GRASS programs
and completely replace BASH as shell for GRASS. What we would get out
of this is:

1. Tab completion for GRASS programs
2. Python as a scripting language (no problems "shelling out" to C or Fortran)
3. IPython is already thread-safe for wxPython GUIs
4. A lot of math tools for plotting and math are already using IPython
as a shell
5. Very powerful introspection capabilities for programs.
6. Windows, Unix, Mac all supported and mature.
7. Future of IPython is network aware for clustered use of GRASS!

The list is too long...

Take a look:

http://ipython.scipy.org/

What do you think?

I'd be willing to start writing the software to "wrap" our existing
programs so that they work with IPython interactively.

--
David Finlayson

[David_Finlayson]

I should add, that one of IPython's features is "autocall". This
removes the need to use parethesis in function calls. What is cool
about this is that we could write wrappers around the grass programs
as functions and within IPython, the syntax of calling the functions
would be virtually identical with the current documentation:

For example:

class general:
     def list(type, mapset):
          print os.popen('g.list type=%, mapset=%' % (type, mapset))

Would be called like this in normal Python:

import grass
g = grass.general()
g.list(type, mapset)

....

In IPython, this could be:

import grass
g = grass.general()
g.list type, mapset

...

That looks almost identical to the current bash command!

[Wolf]

On Thu, 8 Jun 2006, David Finlayson wrote:

What do you think?

Sounds promising, but, personally I'd hate it. Why?

<rant>
First off one of the coolest things with grass is that I get my trusted old tcsh (which I have been using since 1996), I was delighted. This means that to do many things I don't need to learn anything new. I can use env variables and loops and whatnot, all things that I'm used to. I do realize that for people in mac or windows worlds or people new to computers don't know all this and for them it would be all the same. Secondly I'd need to learn python to do any scripting, which sucks. Thirdly I like to have all the gazillion of command-line programs that I have, to filter and do all kinds of magic from withing GRASS. I can also easily launch any non-grass application WITHOUT LEAVING THE GRASS SHELL. And my final argument is: Why would we re-invent the wheel? Command-line shells have been around for a very long time which means that a lot of effort has gone into them, and to be frank we don't have that many resources that we should use it on re-inventing the wheel.
</rant>

--W

--

<:3 )---- Wolf Bergenheim ----( 8:>

[Glynn_Clements1]

David Finlayson wrote:

I should add, that one of IPython's features is "autocall". This
removes the need to use parethesis in function calls. What is cool
about this is that we could write wrappers around the grass programs
as functions and within IPython, the syntax of calling the functions
would be virtually identical with the current documentation:

For example:

class general:
     def list(type, mapset):
          print os.popen('g.list type=%, mapset=%' % (type, mapset))

You don't want to be using a popen()-style interface which takes an
entire command as a string and uses the shell to decompose it, but
something which takes a list of arguments. Using the shell is a
nuisance if any of the arguments need to contain spaces or shell
metacharacters.

--
Glynn Clements <glynn@gclements.plus.com>

[Joel_Pitt]

---------- Forwarded message ----------
From: Joel Pitt <joel.pitt@gmail.com>
Date: Jun 9, 2006 11:50 AM
Subject: Re: [GRASS-dev] A portible shell for GRASS 7+ ?
To: Wolf Bergenheim <wolf+grass@bergenheim.net>

I can see both sides and their benefits.

But I don't think we should force people to use a python shell since
alot of GRASS users are bash/tcsh wizards. I think we should combine
some ideas here.

In the thread "Python GUI toolkits" David mentioned an R or Matlab
style GUI - I like this, I also like the idea of being able to use
python to throw up custom graphs of GIS data (although I know R can do
this as well).

Perhaps we could create a GUI that allowed you to have multiple shells
open, R for GRASS, a bash shell, an IPython one.

Come to think of it there isn't any reason why these shells can't
exist on their own...

Hmm, seems I've come to the end of my train of thought.

-joel

On 6/9/06, Wolf Bergenheim <wolf+grass@bergenheim.net> wrote:

On Thu, 8 Jun 2006, David Finlayson wrote:
>
> What do you think?
>

Sounds promising, but, personally I'd hate it. Why?

<rant>
First off one of the coolest things with grass is that I get my trusted
old tcsh (which I have been using since 1996), I was delighted. This means
that to do many things I don't need to learn anything new. I can use env
variables and loops and whatnot, all things that I'm used to. I do realize
that for people in mac or windows worlds or people new to computers don't
know all this and for them it would be all the same. Secondly I'd need to
learn python to do any scripting, which sucks. Thirdly I like to have all
the gazillion of command-line programs that I have, to filter and do all
kinds of magic from withing GRASS. I can also easily launch any non-grass
application WITHOUT LEAVING THE GRASS SHELL. And my final argument is: Why
would we re-invent the wheel? Command-line shells have been around for a
very long time which means that a lot of effort has gone into them, and to
be frank we don't have that many resources that we should use it on
re-inventing the wheel.
</rant>

--
-Joel

"Wish not to seem, but to be, the best."
                -- Aeschylus

--
-Joel

"Wish not to seem, but to be, the best."
                -- Aeschylus

[David_Finlayson]

Wolf, Joel:

<rant>...</rant>

OK, in retrospect proposing a new shell for GRASS 7 was provocative. I
don't want that kind pressure for a hobby project. Consider this a
proposal for an optional download.

Besides, you're preaching to the choir...search the mailing lists for
my name and command line and you will find at least several rants of
my own on this topic. I need full scripting capability for GRASS and
access to Unix and custom programs I have written for my projects. I
would never compromise that.

So, why am I proposing a Python shell for GRASS?

1. I am intrigued by the possibility of making a Matlab of GIS. GRASS
is the perfect candidate for this. Python and the IPython interpreter
have all the tools needed to make a higher-level shell. Think of this
as the exact compliment to Qgis. Where Qgis wants to make a GIS with
lowest possible barrier to entry, I want to make a GIS with the
highest possible productivity for intermediate to advanced users:

* Imagine having a command line that auto completes options, layers,
raster and vector names!

* How about a display monitor that could be controlled by the command
line or the GUI at the same time (a la Matlabs graphics figures)?

* How about having complete access to the GRASS API via Python SWIG?

* All in the same shell?

Bottom line though is that if I want some creative control over how
the GRASS CLI evolves, I need to shut up and code.

2. Windows GRASS needs this badly. The DOS shell is OK for some things
but Windows comes with no tools for working with text, or databases,
etc., etc., etc. Python provides a lot of power through its libraries
and it is very easy to extend. Using IPython as a shell would really
improve the tool set of native GRASS on Windows with no need for
emulating a Unix environment.

3. I think a lot of GUI-only people would consider the command line if
it was a little easier to learn and use. A GRASS IDE (if you will),
might be really cool. It might be powerful, too.

[Wolfgang]

Hi all,

I'm not an developer but a user of grass and I'm working on grass for cygwin and I would prefer to get rid of cygwin, because I only need it to run grass. But I don't think the windows command line is extendable in a way that it is fully compatible with bash (even if all packages from gnuwin32 (gnuwin32.sf.net) are installed). So I would like to have an alternative shell for grass.

Cheers
Wolfgang

David Finlayson schrieb:

Wolf, Joel:

<rant>...</rant>

OK, in retrospect proposing a new shell for GRASS 7 was provocative. I
don't want that kind pressure for a hobby project. Consider this a
proposal for an optional download.

Besides, you're preaching to the choir...search the mailing lists for
my name and command line and you will find at least several rants of
my own on this topic. I need full scripting capability for GRASS and
access to Unix and custom programs I have written for my projects. I
would never compromise that.

So, why am I proposing a Python shell for GRASS?

1. I am intrigued by the possibility of making a Matlab of GIS. GRASS
is the perfect candidate for this. Python and the IPython interpreter
have all the tools needed to make a higher-level shell. Think of this
as the exact compliment to Qgis. Where Qgis wants to make a GIS with
lowest possible barrier to entry, I want to make a GIS with the
highest possible productivity for intermediate to advanced users:

* Imagine having a command line that auto completes options, layers,
raster and vector names!

* How about a display monitor that could be controlled by the command
line or the GUI at the same time (a la Matlabs graphics figures)?

* How about having complete access to the GRASS API via Python SWIG?

* All in the same shell?

Bottom line though is that if I want some creative control over how
the GRASS CLI evolves, I need to shut up and code.

2. Windows GRASS needs this badly. The DOS shell is OK for some things
but Windows comes with no tools for working with text, or databases,
etc., etc., etc. Python provides a lot of power through its libraries
and it is very easy to extend. Using IPython as a shell would really
improve the tool set of native GRASS on Windows with no need for
emulating a Unix environment.

3. I think a lot of GUI-only people would consider the command line if
it was a little easier to learn and use. A GRASS IDE (if you will),
might be really cool. It might be powerful, too.

[Laurent_C]

2006/6/9, Wolfgang <wollez@gmx.net>:

Hi all,

I’m not an developer but a user of grass and I’m working on grass for
cygwin and I would prefer to get rid of cygwin, because I only need it
to run grass. But I don’t think the windows command line is extendable
in a way that it is fully compatible with bash (even if all packages
from gnuwin32 (gnuwin32.sf.net) are installed). So I would like to have
an alternative shell for grass.

Cheers
Wolfgang

Hello,

Bash isn’t in gnuwin32 packages, but it run natively under Windows.
http://www.steve.org.uk/Software/bash/

Laurent

[Wolfgang]

If I understand correctly, the only limit is the X-server to run Grass on native windows?
Wolfgang

Laurent C. schrieb:

2006/6/9, Wolfgang <wollez@gmx.net <mailto:wollez@gmx.net>>:

    Hi all,

    I'm not an developer but a user of grass and I'm working on grass for
    cygwin and I would prefer to get rid of cygwin, because I only need it
    to run grass. But I don't think the windows command line is
    extendable
    in a way that it is fully compatible with bash (even if all packages
    from gnuwin32 (gnuwin32.sf.net <http://gnuwin32.sf.net>) are
    installed). So I would like to have
    an alternative shell for grass.

    Cheers
    Wolfgang

Hello,

Bash isn't in gnuwin32 packages, but it run natively under Windows.
http://www.steve.org.uk/Software/bash/

Laurent

[Laurent_C]

2006/6/9, Wolfgang Zillig <wollez@gmx.net>:

If I understand correctly, the only limit is the X-server to run Grass
on native windows?
Wolfgang

AFAIK, yes. That why a lot of work is made to remplace the actual tool which need X11, like v.digit and so on. For example, Radim made a qgis-grass binary for windows which work quite well.

Laurent C. schrieb:

2006/6/9, Wolfgang <wollez@gmx.net mailto:[wollez@gmx.net](mailto:wollez@gmx.net)>:

Hi all,

I’m not an developer but a user of grass and I’m working on grass for
cygwin and I would prefer to get rid of cygwin, because I only need it
to run grass. But I don’t think the windows command line is
extendable
in a way that it is fully compatible with bash (even if all packages
from gnuwin32 (gnuwin32.sf.net <http://gnuwin32.sf.net>) are
installed). So I would like to have
an alternative shell for grass.

Cheers
Wolfgang

Hello,

Bash isn’t in gnuwin32 packages, but it run natively under Windows.
http://www.steve.org.uk/Software/bash/
<http://www.steve.org.uk/Software/bash/>

Laurent

---

grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

[cmbarton]

I’ve run the current GIS Manager in Aqua TclTk with the bash terminal and no x11 using Lorenzo’s binaries. It runs fine as long as you don’t start something that requires x11 (e.g. i.points, v.digit, d.profile, nviz). The new georectifier and profiler work without x11, and so should work fine.

If someone could compile the existing Windows native code with the d.modules and gis.m intact, and start it up using native Windows wish it should run theoretically. But I don’t know how to do this and so can’t test it. It would be good to know if our efforts in this direction are helpful, though.

Michael

---

Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics and Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

---

From: “Laurent C.” lrntct@gmail.com
Date: Fri, 9 Jun 2006 14:14:25 +0200
To: Wolfgang Zillig wollez@gmx.net
Cc: grass-devel grass-dev@grass.itc.it
Subject: Re: [GRASS-dev] A portible shell for GRASS 7+ ?

2006/6/9, Wolfgang Zillig wollez@gmx.net:

If I understand correctly, the only limit is the X-server to run Grass
on native windows?
Wolfgang

AFAIK, yes. That why a lot of work is made to remplace the actual tool which need X11, like v.digit and so on. For example, Radim made a qgis-grass binary for windows which work quite well.

Laurent C. schrieb:

2006/6/9, Wolfgang <wollez@gmx.net mailto:wollez@gmx.net>:

Hi all,

I’m not an developer but a user of grass and I’m working on grass for
cygwin and I would prefer to get rid of cygwin, because I only need it
to run grass. But I don’t think the windows command line is
extendable
in a way that it is fully compatible with bash (even if all packages
from gnuwin32 (gnuwin32.sf.net http://gnuwin32.sf.net http://gnuwin32.sf.net) are
installed). So I would like to have
an alternative shell for grass.

Cheers
Wolfgang

Hello,

Bash isn’t in gnuwin32 packages, but it run natively under Windows.
http://www.steve.org.uk/Software/bash/
http://www.steve.org.uk/Software/bash/

Laurent

---

grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

[Glynn_Clements1]

Michael Barton wrote:

If someone could compile the existing Windows native code with the d.modules
and gis.m intact, and start it up using native Windows wish it should run
theoretically. But I don¹t know how to do this and so can¹t test it. It
would be good to know if our efforts in this direction are helpful, though.

I can get most of GRASS to compile on Windows without Cygwin (this
relies upon the not-yet-committed changes to libraster to allow
driver-less rendering).

However, there appear to be some fundamental problems with the MSVC
run-time, e.g. fseek() doesn't work on files opened for update
("wb+"), so I wouldn't expect it to run.

FWIW, the portions which fail to compile are:

  lib/fonts/for_grass
  lib/vector/diglib

Both due to fseek() issues.

  display/d.colors
  raster/r.le/r.le.trace

Undefined reference to sleep()

  display/d.mon/pgms

Undefined reference to ttyname()

  vector/v.digit

Undefined reference to gettimeofday()

  imagery/i.ortho.photo/photo.2image
  imagery/i.ortho.photo/photo.2target
  imagery/i.points
  imagery/i.vpoints

Missing <sys/wait.h>

  imagery/i.class

Undefined SIGTSTP

  imagery/i.ortho.photo/photo.rectify
  imagery/i.rectify

Both define a symbol "compress" which conflicts with zlib.

  display/d.paint.labels
  raster/r.support/modcats
  raster/r.support/modcolr
  raster/r.support/modhist
  raster/r.support/modhead

Attempting to use ln/mv on files which don't exist (need to add the
.exe suffix).

This is with only mandatory external dependencies; no Tcl/Tk, PNG,
TIFF, OpenGL etc.

--
Glynn Clements <glynn@gclements.plus.com>

[cmbarton]

Although this sounds grim, it actually may not be as bad as it seems. Most
are x11 issues, and many have new TclTk equivalents. See below.

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics and Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: Glynn Clements <glynn@gclements.plus.com>
Date: Fri, 9 Jun 2006 21:28:22 +0100
To: Michael Barton <michael.barton@asu.edu>
Cc: "Laurent C." <lrntct@gmail.com>, Wolfgang Zillig <wollez@gmx.net>,
grass-devel <grass-dev@grass.itc.it>
Subject: Re: [GRASS-dev] A portible shell for GRASS 7+ ?

Michael Barton wrote:

If someone could compile the existing Windows native code with the d.modules
and gis.m intact, and start it up using native Windows wish it should run
theoretically. But I don¹t know how to do this and so can¹t test it. It
would be good to know if our efforts in this direction are helpful, though.

I can get most of GRASS to compile on Windows without Cygwin (this
relies upon the not-yet-committed changes to libraster to allow
driver-less rendering).

However, there appear to be some fundamental problems with the MSVC
run-time, e.g. fseek() doesn't work on files opened for update
("wb+"), so I wouldn't expect it to run.

FWIW, the portions which fail to compile are:

lib/fonts/for_grass
lib/vector/diglib

Both due to fseek() issues.

These may be the most problematic

display/d.colors
raster/r.le/r.le.trace

Undefined reference to sleep()

Neither needed for most GIS work.
d.colors requires x11 anyway and would not run without Cygwin

display/d.mon/pgms

Undefined reference to ttyname()

Don't know what this might do

vector/v.digit

Undefined reference to gettimeofday()

Requires x11 and wouldn't run anyway without Cygwin

imagery/i.ortho.photo/photo.2image
imagery/i.ortho.photo/photo.2target
imagery/i.points
imagery/i.vpoints

Missing <sys/wait.h>

i.points and v.points require x11 and replaced by TclTk georectifier. Have
to do without i.ortho.photo, but this is a specialized application.

imagery/i.class

Undefined SIGTSTP

It would be nice if this worked.

imagery/i.ortho.photo/photo.rectify
imagery/i.rectify

i.rectify needs to work for the georectifier, but doesn't need to run in x11
interactive mode.

Both define a symbol "compress" which conflicts with zlib.

display/d.paint.labels
raster/r.support/modcats
raster/r.support/modcolr
raster/r.support/modhist
raster/r.support/modhead

Attempting to use ln/mv on files which don't exist (need to add the
.exe suffix).

d.paint.lables/d.labels replaced by TclTk equivalent in gis.m. Didn't work
right in TclTk canvas anyway (at least on Mac)
r.support would be nice but not critical

This is with only mandatory external dependencies; no Tcl/Tk, PNG,
TIFF, OpenGL etc.

--
Glynn Clements <glynn@gclements.plus.com>

[Glynn_Clements1]

Michael Barton wrote:

> I can get most of GRASS to compile on Windows without Cygwin (this
> relies upon the not-yet-committed changes to libraster to allow
> driver-less rendering).
>
> However, there appear to be some fundamental problems with the MSVC
> run-time, e.g. fseek() doesn't work on files opened for update
> ("wb+"), so I wouldn't expect it to run.
>
> FWIW, the portions which fail to compile are:
>
> lib/fonts/for_grass
> lib/vector/diglib
>
> Both due to fseek() issues.

These may be the most problematic

Definitely.

The lib/fonts/for_grass issue is that splitfont() doesn't work.
lib/vector/diglib fails because the portable I/O test fails. IOW,
these are both run-time failures. It's likely that everything which
uses fseek() on read-write files will fail similarly (I've written
test programs to verify the fseek() behaviour, and they fail in the
same way).

> display/d.mon/pgms
>
> Undefined reference to ttyname()

Don't know what this might do

If the fifth field in the monitorcap file is non-empty, you can only
start the monitor from the specified terminal (i.e. mon.start's stdin
must refer to the specified terminal).

This has a very specific purpose: if you are running GRASS on a pair
of actual hardware terminals (one text terminal and one graphics
terminal, e.g. a DEC vt220 and a Tektronix 4105) on a multi-user
system, it stops you from starting someone else's monitor by mistake.

Given that the Tektronix 4105 driver isn't supported in 5.x, let alone
6.x, I think it's safe to consider this feature "obsolete".

> imagery/i.class
>
> Undefined SIGTSTP

It would be nice if this worked.

It wouldn't be hard to just disable that code with
"#ifndef __MINGW32__". It's catching that signal because curses messes
with the terminal, so you need to either disable Ctrl-Z or catch it
and restore the terminal settings. That isn't applicable to Windows,
as you can't suspend console applications.

> imagery/i.ortho.photo/photo.rectify
> imagery/i.rectify

i.rectify needs to work for the georectifier, but doesn't need to run in x11
interactive mode.

We just need to rename compress() to something which doesn't conflict
with zlib. Actually, compress() is just a stub, so we could probably
just remove it.

> display/d.paint.labels
> raster/r.support/modcats
> raster/r.support/modcolr
> raster/r.support/modhist
> raster/r.support/modhead
>
> Attempting to use ln/mv on files which don't exist (need to add the
> .exe suffix).

d.paint.lables/d.labels replaced by TclTk equivalent in gis.m. Didn't work
right in TclTk canvas anyway (at least on Mac)
r.support would be nice but not critical

Platform.make.in actually has a variable EXE_SUFFIX, but nothing sets it.

--
Glynn Clements <glynn@gclements.plus.com>