[GRASS-dev] [GRASS GIS] #928: 6.4: all shell scripts broken from command line + tcl; wx addon scripts broken

GDAL · February 13, 2010, 11:22am

#928: 6.4: all shell scripts broken from command line + tcl; wx addon scripts
broken
---------------------+------------------------------------------------------
Reporter: hamish | Owner: grass-dev@lists.osgeo.org
Type: defect | Status: new
Priority: blocker | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Keywords: addons | Platform: Linux
Cpu: x86-64 |
---------------------+------------------------------------------------------
6.4 only: (latest svn)

running scripts held in $GRASS_ADDON_PATH from the terminal prompt, with
GRASS_GUI=wxpython never returns.

see this mailing list thread for more from Kim:
http://thread.gmane.org/gmane.comp.gis.grass.devel/37887/focus=38143

worse, with GRASS_GUI=tcltk *all* script modules fail with "`sh:
$GRASS_WISH: command not found`", both official and addon.

When running the addon module from the real terminal prompt with
GRASS_GUI=wxpython you get the following error once for every time you
press the [Stop] button.

{{{
G64:addons > which r.out.gmt
/usr/local/src/grass/addons/r.out.gmt

Traceback (most recent call last):
   File "/usr/local/src/grass/svn/releasebranch_6_4/dist.x86_64-unknown-
linux-gnu/etc/wxpython/gui_modules/goutput.py", line 486, in OnCmdAbort
     self.cmdThread.abort()
   File "/usr/local/src/grass/svn/releasebranch_6_4/dist.x86_64-unknown-
linux-gnu/etc/wxpython/gui_modules/goutput.py", line 108, in abort
     self.requestCmd.abort()
AttributeError: 'CmdThread' object has no attribute 'requestCmd'
}}}

If run from the wxGUI Cmd> prompt with "r.out.gmt --ui", when you click
the window dressing "X" to close the window you get it again in the main
Layer Manager output tab, except this time with an additional popup error
message:
{{{
__Error in command execution r.out.[snip]__

   Execution failed: 'r.out.gmt'
   Details:
   Error:
}}}

If I restart GRASS in tcltk mode and try "r.out.gmt" from the real
terminal prompt something even weirder happens:

{{{
G64:~ > r.out.gmt
sh: $GRASS_WISH: command not found
G64:~ > r.out.gmt --ui
sh: $GRASS_WISH: command not found

G64:~ > $GRASS_WISH
[wish window opens, then I manually "X" it closed]
% G64:~ >
}}}

so $GRASS_WISH is being quoted there once too many times...

I get the same thing if I run "r.out.gmt --ui" from the lower frame in the
Output window.

(can we label that tcl command prompt frame somehow? it is impossible to
guess that typing something there does anything. perhaps sacrifice one of
the 3 input lines for a "Command:" label?)

in 6.5 it all works fine AFAICT.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 18, 2010, 1:56am

#928: 6.4: all shell scripts broken from command line + tcl; wx addon scripts
broken
----------------------+-----------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: blocker | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
----------------------+-----------------------------------------------------
Comment (by hamish):

Replying to [ticket:928 hamish]:
> 6.4 only: (latest svn)
...
> worse, with GRASS_GUI=tcltk *all* script modules fail with
> "`sh: $GRASS_WISH: command not found`", both official and addon.

(but from gis.m they run ok)

> in 6.5 it all works fine AFAICT.

I guess it has to do with this, but unless popen() has different quoting
needs I don't see the difference:

r40849 (relbr64) `Replace G_popen() with popen() (backport from devbr6,
r40834)`

does this need to be backported to 6.x?:

r40904 (trunk) `G_spawn_ex() expects descriptors as "int"s, not
"char*".`

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:1>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 18, 2010, 9:16am

#928: 6.4: all shell scripts broken from command line + tcl; wx addon scripts
broken
----------------------+-----------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: blocker | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
----------------------+-----------------------------------------------------
Comment (by glynn):

Replying to [comment:1 hamish]:

> I guess it has to do with this, but unless popen() has different quoting
needs I don't see the difference:
>
> r40849 (relbr64) `Replace G_popen() with popen() (backport from devbr6,
r40834)`
>
>
> does this need to be backported to 6.x?:
>
> r40904 (trunk) `G_spawn_ex() expects descriptors as "int"s, not
"char*".`

It can't be; the code which it fixes doesn't exist in 6.4 (and isn't
actually used in 6.5, and probably shouldn't have been back-ported).

But that's not the problem here. The problem is r40716, which back-ports
r40713. This was fixed in 6.5 with r40724, which also needs to be back-
ported.

The various "popen"-related issues which appear to be getting conflated
are:

1. The [G_]popen() of $GRASS_WISH for the parameter dialogs. Originally,
$GRASS_WISH wasn't quoted, which may create problems on Windows if it's
somewhere within `$ProgramFiles`. So r40713 quotes it, but I screwed up
(the Unix version used single quotes instead of double quotes, and wasn't
tested). This was fixed in r40724. It isn't applicable to 7.0, as
GRASS_GUI=tcltk isn't supported there.

2. Eliminating (the old) G_popen() in favour of popen(). G_popen() just
mimics popen() on Unix and doesn't work on Windows (which actually
provides a Unix-style popen()).

3. In 7.0, G_popen() is implemented on top of G_spawn_ex() and has a
completely different interface: it accepts an argument list (char**)
rather than a string to be interpreted by the shell. Currently, it's only
used by G_pager() (interface to $GRASS_PAGER) and the code which emails
error messages to the user. In 6.x, it isn't used at all; with popen()
being eliminated, lib/gis/popen.c can (and should) be removed.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:2>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 18, 2010, 11:18am

#928: 6.4: wx addon scripts never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Changes (by hamish):

  * priority: blocker => critical
  * summary: 6.4: all shell scripts broken from command line + tcl; wx
              addon scripts broken => 6.4: wx addon scripts
              never return

Comment:

Replying to [comment:2 glynn]:
> But that's not the problem here. The problem is r40716, which
> back-ports r40713. This was fixed in 6.5 with r40724, which also
> needs to be back-ported.

ok, done. "`sh: $GRASS_WISH: command not found`" is fixed.

> In 6.x, it isn't used at all; with popen() being eliminated,

(I guess you mean G_popen() there)

> lib/gis/popen.c can (and should) be removed.

I am not very happy about non-necessary API changes / dropping of long
standing functions in the stable branch as we don't know what 3rd party
modules are relying on. Adding `G_warning("oi! stop using this function!
use bar() instead!")` is more productive IMO.
In trunk anything goes, but..

----

Still broken:
* add-on scripts run through the wx module GUI in 6.4 finish but never
return. when you hit the [Stop] button you get this error in the term:
`AttributeError: 'CmdThread' object has no attribute 'requestCmd'`

Outstanding wish:
* Tcl/Tk gis.m output window: can we label that command prompt frame
somehow? it is impossible to guess that typing something there does
anything. perhaps sacrifice one of the 3 input lines for a "Command:"
label?

thanks,
Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:3>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 18, 2010, 12:56pm

#928: 6.4: wx addon scripts never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Comment (by glynn):

Replying to [comment:3 hamish]:

> > In 6.x, it isn't used at all; with popen() being eliminated,
>
> (I guess you mean G_popen() there)

Yes.

> > lib/gis/popen.c can (and should) be removed.
>
> I am not very happy about non-necessary API changes / dropping of long
standing functions in the stable branch as we don't know what 3rd party
modules are relying on. Adding `G_warning("oi! stop using this function!
use bar() instead!")` is more productive IMO.

Anything using G_popen() was broken, as it didn't work on Windows (and
offered no advantage to using popen()).

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:4>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 18, 2010, 3:45pm

#928: 6.4: wx addon scripts never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Changes (by kimbesson):

* cc: kimbesson (added)

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:5>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 18, 2010, 9:06pm

#928: 6.4: wx addon scripts never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Comment (by hamish):

would backporting 'g.parser -s' from 6.5 help?

(please document -s in both the help page and add to the --help usage
text)

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:6>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 19, 2010, 11:49am

#928: 6.4: wx addon scripts never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Comment (by glynn):

Replying to [comment:6 hamish]:

> (please document -s in both the help page and add to the --help usage
text)

Done in r41119.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:7>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 19, 2010, 9:51pm

#928: 6.4: wx addon scripts never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Comment (by neteler):

Replying to [comment:7 glynn]:
> Replying to [comment:6 hamish]:
>
> > (please document -s in both the help page and add to the --help usage
text)
>
> Done in r41119.

Backported to 6.5 in r41122.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:8>
GRASS GIS <http://grass.osgeo.org>

Kim_Besson1 · February 22, 2010, 1:41pm

I have just installed laters 6.4 snapshot and it’s still not working. I mean, I can only launch script frame with --ui and it doesn’t stop. At the end, it never exits the cycle.

Kim

2010/2/19 Kim Besson <kimbesson1981@gmail.com>

Hi Hamish
This message is for me? Is it possible to have a GRASS6.4 and run g.parser from 6.5 ?

Kim

2010/2/18 GRASS GIS <trac@osgeo.org>

#928: 6.4: wx addon scripts never return
-----------------------±---------------------------------------------------
Reporter: hamish | Owner: grass-dev@lists.osgeo.org
Type: defect | Status: new
Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
Platform: Linux | Cpu: x86-64
-----------------------±---------------------------------------------------

Comment (by hamish):

would backporting ‘g.parser -s’ from 6.5 help?

(please document -s in both the help page and add to the --help usage
text)

Hamish

–
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:6>

GRASS GIS <http://grass.osgeo.org>

GDAL · March 7, 2010, 7:26pm

#928: 6.4: wx addon scripts never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Comment (by neteler):

Should the g.parser version from 6.5 be backported to 6.4?

Markus

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/928#comment:9>
GRASS GIS <http://grass.osgeo.org>

GDAL · March 12, 2010, 12:59pm

#928: 6.4: wx addon scripts never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Comment (by hamish):

Markus:
> Should the g.parser version from 6.5 be backported to 6.4?

(now done by MarkusN, r41386)

running r.out.gmt shell script addon from wx module GUI tested with
`g.gisenv set="GRASS_GUI=wxpython"`; it still never returns after module
completion with 'requestCmd' Traceback error in the terminal.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:10>
GRASS GIS <http://grass.osgeo.org>

GDAL · March 15, 2010, 2:05am

#928: 6.4: wx module GUIs never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Changes (by hamish):

* summary: 6.4: wx addon scripts never return => 6.4: wx module GUIs
never return

Comment:

see also #1010; it also some built in C modules from the command line.

tested
  * d.rast - never returned.
  * r.info - returned.
  * r.cats - returned.
  * `d.erase --ui` - never returned.
  * `v.out.ascii` + roads (no output) - returns.
  * `v.report opt=print` - returns.

???

Hamish

ps - `v.out.ascii input=roads --ui` fails to pre-seed the GUI as it does
with tcktk.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:11>
GRASS GIS <http://grass.osgeo.org>

GDAL · March 15, 2010, 12:19pm

#928: 6.4: wx module GUIs never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: martinl
      Type: defect | Status: assigned
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Changes (by martinl):

* cc: grass-dev@lists.osgeo.org (added)
* owner: grass-dev@lists.osgeo.org => martinl
* status: new => assigned

Comment:

Replying to [comment:11 hamish]:
> see also #1010; it also some built in C modules from the command line.
>
> tested
   * d.rast - never returned.
   * r.info - returned.
   * r.cats - returned.
   * `d.erase --ui` - never returned.
   * `v.out.ascii` + roads (no output) - returns.
   * `v.report opt=print` - returns.

Please try r41442. Martin

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:12>
GRASS GIS <http://grass.osgeo.org>

GDAL · March 15, 2010, 2:15pm

#928: 6.4: wx module GUIs never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: martinl
      Type: defect | Status: assigned
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Comment (by martinl):

Replying to [comment:11 hamish]:

> ps - `v.out.ascii input=roads --ui` fails to pre-seed the GUI as it does
with tcktk.

hopefully fixed in r41447 (trunk) and r41448 (devbr6). I would suggest to
backport this change to relbr64 after 6.4.0. It needs some testing and can
affect wxGUI quite significantly.

Martin

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:13>
GRASS GIS <http://grass.osgeo.org>

GDAL · March 16, 2010, 8:18am

#928: 6.4: wx module GUIs never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: martinl
      Type: defect | Status: assigned
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Comment (by hamish):

Replying to [comment:12 martinl]:
> * d.rast - never returned.
...
> Please try r41442. Martin

thanks; works for me (including r.out.gmt addon).

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:14>
GRASS GIS <http://grass.osgeo.org>

GDAL · March 16, 2010, 8:24am

#928: 6.4: wx module GUIs never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: martinl
      Type: defect | Status: assigned
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Comment (by hamish):

Replying to [comment:13 martinl]:
> Replying to [comment:11 hamish]:
>
> > ps - `v.out.ascii input=roads --ui` fails to pre-seed the GUI
> > as it does with tcktk.
>
> hopefully fixed in r41447 (trunk) and r41448 (devbr6).

thanks, works for me with a couple of minor points:
- d.erase only works if color != white. (then answer disappears from
command at bottom of window & I guess the command becomes recursive..?)

- "`v.out.ascii roads fs=, --ui`": fs=, gets skipped but other values
work. (I guess a syntax thing)

> I would suggest to backport this change to relbr64 after 6.4.0.
> It needs some testing and can affect wxGUI quite significantly.

fine with me. the main point in supporting this is to give better control
to the menus. (pre-tick non-interactive mode, etc)

TODO: verify 6.4 wingrass with python addon script and the newly
backported g.parser -s.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:15>
GRASS GIS <http://grass.osgeo.org>

GDAL · April 8, 2010, 7:50am

#928: 6.4: wx module GUIs never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: martinl
      Type: defect | Status: assigned
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Comment (by marisn):

g.gui in WinGRASS-6.4.SVN-r41749-1 is also a blocker. Running g.gui
gui=wxpython locks CMD till exit from GUI. Same with gui=tcltk. Still it's
possible to launch old tcltk GUI with gis.m without blocking CMD. 1:0 in
favor of gis.m.

Also g.gui -u wxpython has no effect on modules launched from CMD - I
still get tcltk GUI afterwards. g.gisenv displays "GRASS_GUI=wxpython"

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/928#comment:16>
GRASS GIS <http://grass.osgeo.org>

GDAL · April 8, 2010, 8:09am

#928: 6.4: wx module GUIs never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: martinl
      Type: defect | Status: assigned
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Comment (by hamish):

Replying to [comment:16 marisn]:
> Also g.gui -u wxpython has no effect on modules launched from
> CMD - I still get tcltk GUI afterwards. g.gisenv displays
> "GRASS_GUI=wxpython"

that is due to the way that the icons/menu items are set up on WinGrass.
If you look at the Properties they actually call "`grass64 -tcltk`" and
"`grass64 -wxpython`". Which has the effect of making the selection
permanent. Solution: if you want the tcltk gui click on the tcltk menu
item, if you want the wx gui click on the wx gui menu item. copy either
one to your desktop as desired.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:17>
GRASS GIS <http://grass.osgeo.org>

GDAL · April 8, 2010, 8:47am

#928: 6.4: wx module GUIs never return
-----------------------+----------------------------------------------------
  Reporter: hamish | Owner: martinl
      Type: defect | Status: assigned
  Priority: critical | Milestone: 6.4.0
Component: default | Version: svn-releasebranch64
Resolution: | Keywords: addons
  Platform: Linux | Cpu: x86-64
-----------------------+----------------------------------------------------
Comment (by marisn):

Replying to [comment:17 hamish]:
>that is due to the way that the icons/menu items are set up on WinGrass.
If you look at the Properties they actually call "`grass64 -tcltk`" and
"`grass64 -wxpython`". Which has the effect of making the selection
permanent. Solution: if you want the tcltk gui click on the tcltk menu
item, if you want the wx gui click on the wx gui menu item. copy either
one to your desktop as desired.
>
> Hamish
Ah. That explains a bit, still I was using `grass64 -text`, as it gives me
CMD. Lack of normal terminal is PITA. Also then probably g.gui should
print a warning when launched with -u on windows, as I expect my later
actions to take precedence over startup type.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/928#comment:18>
GRASS GIS <http://grass.osgeo.org>