[GRASS-dev] Re: 'g.gui wxpython' won't work in wingrass as wxgui is a shell script

[cmbarton]

On Feb 28, 2008, at 1:56 PM, grass-dev-request@lists.osgeo.org wrote:

Date: Fri, 29 Feb 2008 00:04:59 +0600
From: Ivan Shmakov <ivan@theory.asu.ru>
Subject: [GRASS-dev] Re: 'g.gui wxpython' won't work in wingrass as
  wxgui is a shell script
To: grass-dev@lists.osgeo.org
Cc: Ivan Shmakov <oneingray@gmail.com>
Message-ID: <m2pruhj6t0.fsf@cherry.siamics.int>
Content-Type: text/plain; charset=us-ascii

Michael Barton <michael.barton@asu.edu> writes:

[...]

I thought the whole reason to have a C-based g.gui was to avoid the
use of *nix specific bash shell scripts for launching GUI's. That way
they would work on all systems without the hack of having to rewrite
*.sh as a *.bat. Why can't the C-code just do the job of the old
shell scripts and directly launch the GUI code in TclTk or wxPython.

The shell scripts are only a convenience anyway, as these could be
launched from the GRASS command line with a bit more complicated
command--e.g., python "$GISBASE/etc/wxpython/wxgui.py" &

The name of the initialization module for each GUI (e.g., wxguy.py in
wxPython) could be set to a variable in init.sh (or its successor) to
avoid hard coding it in C.

  It may be a stupid question, but what scripting language is
  proposed to replace POSIX Shell in GRASS? The GRASS environment
  seems to benefit a lot from relying on a Shell-like language.
  E. g., I could hardly imagine using any language other than
  Shell for the following:

$ g.mlist type=rast pattern=2008-\*-temperature \
     (while read r ; do
          s="$r"-celsius ; r.mapcalc "\"$s\" = \"$r\" - 273.15" ; \
      done)

  (Surely, there's an Olin Shivers et al work [1, 2], but it seems
  to me that Shell is a bit easier to explain to a non-programmer
  than Scheme [3, 4].)

[1] http://www.scsh.net/
[2] http://www.scsh.net/resources/commander-s.html
[3] http://www.r6rs.org/
[4] http://www.schemers.org/

For the foreseeable future, Linux/Unix systems will continue to have a shell available to do the operation in your example. One of the important features of GRASS is that many operations can be performed in a variety of scripting environments--including bash or other shells.

However, Windows does not have a shell. If GRASS is to be truly cross-platform, then all basic GRASS functions need to operate on all supported platforms. This poses a significant problem for Windows users. Cygwin has been quite difficult to install for most users, very difficult to use in a shared classroom/lab setting, and (in my experience) somewhat unstable. There is now a 'native' Windows build of GRASS, but it requires custom hacks for bash scripts and other bashisms common in GRASS--and installing msys--and these have been the most problematic part of running GRASS in Windows so far.

Also, many users are not particularly comfortable with arcane bash shell commands--like in your example above--although I know others who have a special fondness for bash.

The current plan is ultimately for all core GRASS functions to run as binaries in a cross-platform compiled language (C for most things) OR in Python as a cross-platform scripting environment for GUI and 'permanent' scripts (i.e., those that are distributed as part of GRASS). I'm not familiar with Olin Shivers; I've heard of Scheme, but don't know it. However, IMHO, Python is much easier to teach someone and to work with than bash.

Nevertheless, for those on *nix systems, bash will continue to be a scripting platform for GRASS into the near future.

Michael

[Ivan_Shmakov1]

Michael Barton <michael.barton@asu.edu> writes:

[...]

>> It may be a stupid question, but what scripting language is
>> proposed to replace POSIX Shell in GRASS? The GRASS environment
>> seems to benefit a lot from relying on a Shell-like language.
>> E. g., I could hardly imagine using any language other than Shell
>> for the following:

>> $ g.mlist type=rast pattern=2008-\*-temperature \
>> (while read r ; do
>> s="$r"-celsius ; r.mapcalc "\"$s\" = \"$r\" - 273.15" ; \
>> done)

>> (Surely, there's an Olin Shivers et al work [1, 2], but it seems to
>> me that Shell is a bit easier to explain to a non-programmer than
>> Scheme [3, 4].)

>> [1] http://www.scsh.net/
>> [2] http://www.scsh.net/resources/commander-s.html
>> [3] http://www.r6rs.org/
>> [4] http://www.schemers.org/

> For the foreseeable future, Linux/Unix systems will continue to have
> a shell available to do the operation in your example. One of the
> important features of GRASS is that many operations can be performed
> in a variety of scripting environments--including bash or other
> shells.

> However, Windows does not have a shell. If GRASS is to be truly
> cross- platform, then all basic GRASS functions need to operate on
> all supported platforms. This poses a significant problem for Windows
> users. Cygwin has been quite difficult to install for most users,
> very difficult to use in a shared classroom/lab setting, and (in my
> experience) somewhat unstable. There is now a 'native' Windows build
> of GRASS, but it requires custom hacks for bash scripts and other
> bashisms common in GRASS

  As Bash becomes somewhat less popular in the GNU world and being
  replaced by other POSIX-compatible Shells, it makes sense to get
  rid of bashisms irrespective of the platform.

> --and installing msys--and these have been the most problematic part
> of running GRASS in Windows so far.

  Strangely enough, I had no troubles installing Msys recently
  (along with Tcl & Tk I've cross-compiled for W32.)

> Also, many users are not particularly comfortable with arcane bash
> shell commands--like in your example above--although I know others
> who have a special fondness for bash.

> The current plan is ultimately for all core GRASS functions to run as
> binaries in a cross-platform compiled language (C for most things) OR
> in Python as a cross-platform scripting environment for GUI and
> 'permanent' scripts (i.e., those that are distributed as part of
> GRASS). I'm not familiar with Olin Shivers; I've heard of Scheme, but
> don't know it.

  The point is that Scsh, designed by Olin Shivers, employs a
  special Shell-like notation for running external commands. I
  don't know any other language to implement such a feature.

> However, IMHO, Python is much easier to teach someone and to work
> with than bash.

  Well, I've heard of Python, but don't know it. Could you please
  show me the ``g.mlist ...'' fragment above rewritten in Python?

> Nevertheless, for those on *nix systems, bash will continue to be a
> scripting platform for GRASS into the near future.

[Hamish]

Michael:

> bashisms common in GRASS

Ivan:

As Bash becomes somewhat less popular in the GNU world and being
replaced by other POSIX-compatible Shells, it makes sense to get
rid of bashisms irrespective of the platform.

FWIW all known bashisms were removed prior to GRASS 6.2.2.
This is mostly because Ubuntu now uses dash for /bin/sh and a lot of
folks use Ubuntu. Anything left needing bash explicitly uses
#!/bin/bash

Michael has a point in that GRASS heavily relies on UNIX shell scripts,
which is ok for Mac (after fixing GNUisms) but not for MS Windows.
Msys, mingw, and cygwin exist but are in the end all just different
ways of putting lipstick on a pig.

Well, I've heard of Python, but don't know it.

for some things it will be easier, for others not. e.g. for shell
command scripting nothing will ever be as good as sh, but for more
complex programming python will be better. It's just a matter of
choosing a compromise. e.g. for your g.mlist example sh is hard to
beat. But for something complex like r.in.wms or v.in.garmin, sh just
isn't the right tool for the job.

for a quick primer see:
  10 minutes Python tutorial for programmers of other languages
  http://www.poromenos.org/tutorials/python

Hamish

      ____________________________________________________________________________________
Looking for last minute shopping deals?
Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping

[Ivan_Shmakov1]

Hamish <hamish_b@yahoo.com> writes:

>>> bashisms common in GRASS

>> As Bash becomes somewhat less popular in the GNU world and being
>> replaced by other POSIX-compatible Shells, it makes sense to get rid
>> of bashisms irrespective of the platform.

> FWIW all known bashisms were removed prior to GRASS 6.2.2. This is
> mostly because Ubuntu now uses dash for /bin/sh and a lot of folks
> use Ubuntu. Anything left needing bash explicitly uses #!/bin/bash

  I. e., `i.spectral', `r.mapcalculator', `r.tileset' and
  `v.in.gps.babel'? Could someone please point me to the list of
  the identified Shell portability issues of these scripts?

> Michael has a point in that GRASS heavily relies on UNIX shell
> scripts, which is ok for Mac (after fixing GNUisms) but not for MS
> Windows. Msys, mingw, and cygwin exist but are in the end all just
> different ways of putting lipstick on a pig.

>> Well, I've heard of Python, but don't know it.

> for some things it will be easier, for others not. e.g. for shell
> command scripting nothing will ever be as good as sh,

  Well, for the Lisp family of the languages a syntax extension
  mechanism is an inherent feature. Thus, nothing really prevents
  a Lisp from deciphering something like the following:

(like-a-shell
  (| (run "g.mlist" "type=rast" "pattern=2008-\\*-temperature")
     (for-each-line r
       (let ((s (concat r "-celsius")))
         (run "r.mapcalc" "\"$s\" = \"$r\" - 273.15")))))

  Though even such a notation, resembling the Shell code rather
  closely, is hardly acceptable for an interactive use. (Except
  for ones using Emacs as their command line editor.)

> but for more complex programming python will be better. It's just a
> matter of choosing a compromise. e.g. for your g.mlist example sh is
> hard to beat.

  But that's exactly my point -- Python is hardly an alternative
  for the 1.5-liners that you type into the Shell interactively.
  And depriving a GRASS user of a Shell does, in my opinion,
  seriously limit the usefulness of GRASS.

  Thus, while MSYS' or Cygwin's Shell (and the ``usual'' tools)
  may be somewhat difficult to deploy on W32, it seems to me just
  unavoidable to have them installed in order to do anything
  useful with GRASS.

  As an alternative, it may be that GRASS could benefit from a
  Shell (et al)... implemented in Python. It's all but an
  original idea. E. g., Emacs implements Eshell [1], though it
  may be noted that because of Emacs Lisp Emacs dependency on the
  Shell is much looser than that of GRASS.

  Besides, such a Shell would have a much wider field of
  applicability than just being the interactive shell for GRASS.
  I guess, that may make such a project to attract some interest
  from the general Python community.

> But for something complex like r.in.wms or v.in.garmin, sh just isn't
> the right tool for the job.

> for a quick primer see: 10 minutes Python tutorial for programmers of
> other languages http://www.poromenos.org/tutorials/python

[1] http://www.gnu.org/software/emacs/manual/html_mono/eshell.html

[Hamish]

Hamish writes:
> FWIW all known bashisms were removed prior to GRASS 6.2.2.

...

> Anything left needing bash explicitly uses #!/bin/bash

Ivan wrote:

  I. e., `i.spectral', `r.mapcalculator', `r.tileset' and
  `v.in.gps.babel'? Could someone please point me to the list of
  the identified Shell portability issues of these scripts?

$ grep -rI 'bin/bash' scripts/* | cut -f1 -d: | grep -v '/.svn/'

scripts/i.spectral/i.spectral

variable arrays are bash specific:
   ${COORD[$i]}

I am not sure if this is ok or not:
   $( cat "$TMP1")
?

scripts/r.tileset/r.tileset: heavy use of variable arrays

scripts/v.in.gpsbabel/v.in.gpsbabel: not sure
maybe mentioned in the bashism threads in the archives?

ISTR that dash had an option to bulk test a script for bashisms.
(see the archives)

I don't know if it is worth the effort to worry about making those sh
compatible. Even on Ubuntu where !/bin/sh isn't Bash, Bash is still
there from the terminal prompt and if you request it in the shebang.
And we've had no complaints from embedded users.
v.in.gpsbabel and r.tileset should both be rewritten in another
language anyway (python) so spending lots of effort on the current
versions of them would be energy better spent somewhere else IMO.

what's wrong with r.mapcalculator?

  As an alternative, it may be that GRASS could benefit from a
  Shell (et al)... implemented in Python. It's all but an
  original idea. E. g., Emacs implements Eshell [1], though it
  may be noted that because of Emacs Lisp Emacs dependency on the
  Shell is much looser than that of GRASS.

  Besides, such a Shell would have a much wider field of
  applicability than just being the interactive shell for GRASS.
  I guess, that may make such a project to attract some interest
  from the general Python community.

I don't think we should get into the business of trying to reimplement
a full shell. Interesting idea about putting some tasty bait out into
the wider Python developer community.

See also GRASS's tcltk gronsole capability* & QGIS's python shell menu
item.

[*] try typing 'ls' and then click [Run], or 'xeyes' and then [Run
(bg)] in the lower panel of GIS.m's ouput window. I guess we could put
some text like "console:" above the lower frame.

Hamish

      ____________________________________________________________________________________
Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs

[Ivan_Shmakov1]

Hamish <hamish_b@yahoo.com> writes:

>>> FWIW all known bashisms were removed prior to GRASS 6.2.2.

> ...

>>> Anything left needing bash explicitly uses #!/bin/bash

>> I. e., `i.spectral', `r.mapcalculator', `r.tileset' and
>> `v.in.gps.babel'? Could someone please point me to the list of the
>> identified Shell portability issues of these scripts?

> $ grep -rI 'bin/bash' scripts/* | cut -f1 -d: | grep -v '/.svn/'

  BTW, you can use `-l' instead of `cut', e. g.:

   $ grep -rlI 'bin/bash' scripts/* | grep -v '/.svn/'

> scripts/i.spectral/i.spectral

> variable arrays are bash specific: ${COORD[$i]}

> I am not sure if this is ok or not: $( cat "$TMP1") ?

  Completely. See, e. g.:

http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_03

  Furthermore, I'd recommend the $(...) form for all the new code.

> scripts/r.tileset/r.tileset: heavy use of variable arrays

> scripts/v.in.gpsbabel/v.in.gpsbabel: not sure maybe mentioned in the
> bashism threads in the archives?

  It seems that was because of `echo -e' [1] which seems to have
  been subsequently removed. The `checkbashisms' script shows
  nothing for `v.in.gpsbabel' now.

[1] nntp://news.gmane.org/gmane.comp.gis.grass.devel/19389

  It was noted that `echo' is generally ``unsafe''. Consider,
  e. g., the following code:

echo "$a"

  Contrary to the expectations, this code /doesn't/ print the
  value of the `a' variable in all the possible cases. E. g.:

$ echo "$a"
$ printf "a: %s\n" "$a"
a: -n
$

  (And it gets even worse if the `echo' command is extended to
  support `-e' as well.)

  Fortunately, the only case when this ``feature'' may be
  effective is the `echo "$..." ...' case.

> ISTR that dash had an option to bulk test a script for bashisms.
> (see the archives)

  ?

> I don't know if it is worth the effort to worry about making those sh
> compatible. Even on Ubuntu where !/bin/sh isn't Bash, Bash is still
> there from the terminal prompt and if you request it in the shebang.
> And we've had no complaints from embedded users. v.in.gpsbabel and
> r.tileset should both be rewritten in another language anyway
> (python) so spending lots of effort on the current versions of them
> would be energy better spent somewhere else IMO.

  Agreed.

> what's wrong with r.mapcalculator?

  Nothing. It was my mistake.

[...]

> See also GRASS's tcltk gronsole capability* & QGIS's python shell
> menu item.

> [*] try typing 'ls' and then click [Run], or 'xeyes' and then [Run
> (bg)] in the lower panel of GIS.m's ouput window. I guess we could
> put some text like "console:" above the lower frame.

  Unfortunately, it won't allow one to ``[Run]'' `for i in 1 2 3 ;
  do something ; done' if there'd be no Shell.

[Glynn_Clements1]

Ivan Shmakov wrote:

> but for more complex programming python will be better. It's just a
> matter of choosing a compromise. e.g. for your g.mlist example sh is
> hard to beat.

  But that's exactly my point -- Python is hardly an alternative
  for the 1.5-liners that you type into the Shell interactively.
  And depriving a GRASS user of a Shell does, in my opinion,
  seriously limit the usefulness of GRASS.

We aren't depriving them of a shell. We're eliminating the requirement
to install bash even for users who would normally use either the GUI
or cmd.exe.

Oh, and we're also eliminating a source of problems for GRASS
developers. On Windows, the problems with shell scripts begin with the
need to install bash, but they certainly don't end there.

  Thus, while MSYS' or Cygwin's Shell (and the ``usual'' tools)
  may be somewhat difficult to deploy on W32, it seems to me just
  unavoidable to have them installed in order to do anything
  useful with GRASS.

Not really. Simple tasks can be done with just the GUI. More complex
tasks really deserve a proper programming language. The range
inbetween, where bash is a reasonable solution, is actually quite
narrow. Even moreso when you consider that the lower end of that range
is adequately covered by cmd.exe (and most Windows users will probably
be more familiar with that than with bash).

IOW, anything which is too complex for cmd.exe should probably be done
with a real language, even if it can be done with bash.

--
Glynn Clements <glynn@gclements.plus.com>

[Ivan_Shmakov1]

Glynn Clements <glynn@gclements.plus.com> writes:

>>> but for more complex programming python will be better. It's just a
>>> matter of choosing a compromise. e.g. for your g.mlist example sh
>>> is hard to beat.

>> But that's exactly my point -- Python is hardly an alternative for
>> the 1.5-liners that you type into the Shell interactively. And
>> depriving a GRASS user of a Shell does, in my opinion, seriously
>> limit the usefulness of GRASS.

> We aren't depriving them of a shell. We're eliminating the
> requirement to install bash even for users who would normally use
> either the GUI or cmd.exe.

  However, in the other post you're saying that you ``wouldn't
  suggest using cmd.exe for scripts.'' And that's the whole
  point: cmd.exe is a much worser ``programming language'' than
  even POSIX Shell is.

> Oh, and we're also eliminating a source of problems for GRASS
> developers. On Windows, the problems with shell scripts begin with
> the need to install bash, but they certainly don't end there.

  Unfortunately, I have to agree on that.

  However, won't the communication issues arise due to the fact
  that while GRASS/Unix users would use Shell (as it's quite a
  natural choice), while their GRASS/W32 counterparts would be
  advised to choose, e. g., Python?

  In my opinion, a portable implementation of a Shell (and the
  related tools) would be of a much help to the both sides of the
  problem. (And fixing MSYS issues may be an option.) Though
  it's completely out of the scope of the GRASS project.

  Finally, I could only hope of that there'd be more GNU
  installations in the future. (Would there be a little more help
  for me here, I'd certainly prefer running GRASS under GNU/Linux
  in the classroom.)

>> Thus, while MSYS' or Cygwin's Shell (and the ``usual'' tools) may be
>> somewhat difficult to deploy on W32, it seems to me just unavoidable
>> to have them installed in order to do anything useful with GRASS.

> Not really. Simple tasks can be done with just the GUI.

  One certainly won't go far only doing simple tasks.

> More complex tasks really deserve a proper programming language. The
> range inbetween, where bash is a reasonable solution, is actually
> quite narrow.

  The only thing that I have to say in the defense of Bash is that
  the little languages always have a narrow, but not a negligible
  niche.

> Even moreso when you consider that the lower end of that range is
> adequately covered by cmd.exe (and most Windows users will probably
> be more familiar with that than with bash).

  I guess that most of the Windows users which aren't familiar
  with programming aren't probably familiar with cmd.exe either,
  and those who're familiar with the former most probably know a
  programming language which is much better suited for GRASS
  programming than cmd.exe.

> IOW, anything which is too complex for cmd.exe should probably be
> done with a real language, even if it can be done with bash.

  I don't think so.

[Glynn_Clements1]

Ivan Shmakov wrote:

>>> but for more complex programming python will be better. It's just a
>>> matter of choosing a compromise. e.g. for your g.mlist example sh
>>> is hard to beat.

>> But that's exactly my point -- Python is hardly an alternative for
>> the 1.5-liners that you type into the Shell interactively. And
>> depriving a GRASS user of a Shell does, in my opinion, seriously
>> limit the usefulness of GRASS.

> We aren't depriving them of a shell. We're eliminating the
> requirement to install bash even for users who would normally use
> either the GUI or cmd.exe.

  However, in the other post you're saying that you ``wouldn't
  suggest using cmd.exe for scripts.'' And that's the whole
  point: cmd.exe is a much worser ``programming language'' than
  even POSIX Shell is.

That's a false dichotomy. There are many languages which can be used
for scripting, and interactive shells (i.e. cmd.exe or /bin/sh) are
the worst of the available choices.

On Unix, /bin/sh has the advantage that it's guaranteed to be
available. That's its *only* advantage.

On Windows, you don't even have guaranteed availability as a reason to
use /bin/sh. If you're going to install an interpreter for scripts
which you will be writing yourself, you may as well install Python,
Perl, Tcl, VB, or just about anything other than /bin/sh.

Also, many users have no need for non-trivial scripts. cmd.exe is fine
for trivial scripts where you just want to run a canned sequence of
commands without conditionals, loops, string manipulation, etc. It's
equally fine for use as an interactive shell. Such users shouldn't
need a Bourne shell, and wouldn't need one if GRASS didn't make it a
requirement.

> Oh, and we're also eliminating a source of problems for GRASS
> developers. On Windows, the problems with shell scripts begin with
> the need to install bash, but they certainly don't end there.

  Unfortunately, I have to agree on that.

  However, won't the communication issues arise due to the fact
  that while GRASS/Unix users would use Shell (as it's quite a
  natural choice), while their GRASS/W32 counterparts would be
  advised to choose, e. g., Python?

Unix programmers would also also be advised to choose something other
than /bin/sh for scripts. Of course, they're free to use /bin/sh if
they really want to, just as Windows programmers are free to use
cmd.exe (or even to install a Bourne shell, if they're feeling
masochistic).

  In my opinion, a portable implementation of a Shell (and the
  related tools) would be of a much help to the both sides of the
  problem. (And fixing MSYS issues may be an option.) Though
  it's completely out of the scope of the GRASS project.

Unfortunately, Bourne shell is just too "weak" a language to write
portable scripts without going to a lot of trouble. Windows ports of
Bourne shells invariably make some compromises to maintain
compatibility with Unix (e.g. internal filename syntax to avoid the
fact that PATH uses a colon as the separator but Windows uses it for
drive lettors), with the result that you lose compatibility with
Windows (if you pass an MSys filename to a program which then passes
it to the MSVCRT functions, it fails).

Bourne shell sucks as a programming language on Unix, but it sucks
even worse on Windows.

  Finally, I could only hope of that there'd be more GNU
  installations in the future. (Would there be a little more help
  for me here, I'd certainly prefer running GRASS under GNU/Linux
  in the classroom.)

Also, bear in mind that the Cygwin version won't be going anywhere.
That has the advantage that you don't need to deal with the mismatch
between MSys' syntax and Windows syntax, as Cygwin maintains Unix
compatibility all the way down.

>> Thus, while MSYS' or Cygwin's Shell (and the ``usual'' tools) may be
>> somewhat difficult to deploy on W32, it seems to me just unavoidable
>> to have them installed in order to do anything useful with GRASS.

> Not really. Simple tasks can be done with just the GUI.

  One certainly won't go far only doing simple tasks.

> More complex tasks really deserve a proper programming language. The
> range inbetween, where bash is a reasonable solution, is actually
> quite narrow.

  The only thing that I have to say in the defense of Bash is that
  the little languages always have a narrow, but not a negligible
  niche.

It's narrow enough that, for any task where bash is the optimal
solution, either cmd.exe or a real language won't be all that far from
optimal. IOW, it's not a major problem if you don't have it.

> Even moreso when you consider that the lower end of that range is
> adequately covered by cmd.exe (and most Windows users will probably
> be more familiar with that than with bash).

  I guess that most of the Windows users which aren't familiar
  with programming aren't probably familiar with cmd.exe either,
  and those who're familiar with the former most probably know a
  programming language which is much better suited for GRASS
  programming than cmd.exe.

I'm not sure what your point is here. bash has no merit for either set
of users. Users who don't know cmd.exe won't know bash either. Users
who only know a real language should (and probably will) use that.
Windows users who know bash but not cmd.exe are likely to be
non-existent. Windows users who only know cmd.exe but find it
inadequate need to learn another language, and they would be better
off learning anything but bash.

> IOW, anything which is too complex for cmd.exe should probably be
> done with a real language, even if it can be done with bash.

  I don't think so.

Why?

--
Glynn Clements <glynn@gclements.plus.com>