[GRASS-dev] [GRASS GIS] #629: WinGRASS: spaces in pathnames

GRASS GRASS developer
1

#629: WinGRASS: spaces in pathnames
----------------------------+-----------------------------------------------
Reporter: hamish | Owner: grass-dev@lists.osgeo.org
     Type: defect | Status: new
Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Keywords: wingrass, msys | Platform: MSWindows XP
      Cpu: x86-32 |
----------------------------+-----------------------------------------------
Hi,

trying the native stand-alone installer (rev37703) installed in
C:\Program Files\GRASS GIS\GRASS-6-SVN to expose/fix some missing quoting.

The icon to start the wxPython GUI directly works, but the Cmd> prompt
fails for `v.db.select roads where="label ~ 'highway'"` with:

{{{
Bad file descriptor: Files/GRASS
}}}

----

The "GRASS with MSYS" icon doesn't get very far at all due to msys bugs:

in msys.bat :startrxvt section the line
{{{
if NOT EXIST %WD%rxvt.exe goto startsh
}}}

needs to have %WD%rxvt.exe "quoted" otherwise you get this error in the
terminal window: (but it continues anyway)
{{{
The system cannot find the path specified.
}}}

a few lines later in `if NOT "x%*" == "x" start %WD%rxvt -backspacekey ...
-e /bin/sh --login -c "%*"` it fails with a pop up window:
{{{
Windows cannot find 'C:\Program'. Make sure you typed the name correctly,
...
}}}

if you "quote" %WD%rxvt it then gives the error that it can not find the
program `-backspacekey, Make sure that you typed the name correctly, ...`.

maybe using a newer msys package helps fix these?
if not, time to report some bugs to them.

----

also, the grass dirname in msys/etc/fstab needs to be quoted?
(do that in mswindows/GRASS-Installer.nsi)

knock these things off one by one...
Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629&gt;
GRASS GIS <http://grass.osgeo.org>

2

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by hamish):

also I notice the C:/Program Files/GRASS GIS/GRASS-6-SVN/msys/grass/ dir
is empty.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:1&gt;
GRASS GIS <http://grass.osgeo.org>

3

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by hamish):

see
https://sourceforge.net/tracker/index.php?func=detail&aid=1511614&group_id=2435&atid=102435

http://mingw.cvs.sourceforge.net/viewvc/*checkout*/mingw/msys/dvlpr/doc/msys/README.rtf?
content-type=text%2Fplain

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:2&gt;
GRASS GIS <http://grass.osgeo.org>

4

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by hamish):

more links:

- bug roundup:
https://sourceforge.net/search/?group_artifact_id=102435&type_of_search=artifact&group_id=2435&words=spaces

- patch for -backspacekey bug:
https://sourceforge.net/tracker/index.php?func=detail&aid=1840961&group_id=2435&atid=102435

- solve fstab spaces with DOSified pathname: `dir /x` or batch code:
https://sourceforge.net/tracker/index.php?func=detail&aid=1087569&group_id=2435&atid=202435

- mailing list discussion
http://www.nabble.com/make-target---dependency-names-with-spaces-
td22997327.html

after the "start" patch in msys.bat and changing the pathname to
C:/PROGRA~1/GRASSG~1/GRASS-6-SVN in msys/etc/fstab, then I can
successfully start the wxPython GUI.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:3&gt;
GRASS GIS <http://grass.osgeo.org>

5

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by glynn):

Replying to [ticket:629 hamish]:

> trying the native stand-alone installer (rev37703) installed in
> C:\Program Files\GRASS GIS\GRASS-6-SVN to expose/fix some missing
quoting.
>
> The icon to start the wxPython GUI directly works, but the Cmd> prompt
fails for `v.db.select roads where="label ~ 'highway'"` with:
>
{{{
Bad file descriptor: Files/GRASS
}}}

This one isn't an MSys issue, as MSys shouldn't be involved in running the
wxPython GUI, or in running commands from it.

Did the shlex fix make it into the version which you're testing?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:4&gt;
GRASS GIS <http://grass.osgeo.org>

6

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by hamish):

> Replying to [ticket:629 hamish]:
> > The icon to start the wxPython GUI directly works, but the
> > Cmd> prompt fails for
> > `v.db.select roads where="label ~ 'highway'"` with:
> >
{{{
  Bad file descriptor: Files/GRASS
}}}

Replying to [comment:4 glynn]:
> This one isn't an MSys issue, as MSys shouldn't be involved in
> running the wxPython GUI, or in running commands from it.
>
> Did the shlex fix make it into the version which you're testing?

yes.

I also notice that error after fixes to msys.bat I start grass + msys
terminal from C:\Program Files\GRASS GIS\...

looking trough "`set`" I don't see anything obviously broken.

running `v.db.select roads` from the msys command line with DEBUG=5 it
shows that it gets up to
{{{
D2/5: opendir c:/Program Files/GRASS GIS/GRASS-6-SVN\driver\db\
}}}

(slash directions verbatim)

then it locks up the msys terminal.

that debug message comes from lib/db/dbmi_base/dbmscap.c

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:5&gt;
GRASS GIS <http://grass.osgeo.org>

7

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by hamish):

Hi,

the msys maintainers have decided to reject patches that correct spaces-
in-pathnames bugs. i.e. installations to `C:\Program Files\...` just got
that much harder.

https://sourceforge.net/tracker/?func=detail&atid=102435&aid=1511614&group_id=2435

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:6&gt;
GRASS GIS <http://grass.osgeo.org>

8

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by cnielsen):

Bad news about the spaces...

In the new wingrass release (6.4.0rc5 later today) I've updated msys.bat
to:

...{{{ start "MSYS 1.0" "%WD%rxvt" }}}...

I can't think of a way the "dir /x" fix could be used within the installer
to adjust things to the 8.3 path names, but if someone else can I'll
implement it.

The empty grass dir is a mount point for the install directory and allows
the grass with msys shortcuts to be executed with {{{msys.bat
/grass/bin/grass64 -wxpython}}} etc. I tried quotes in the fstab file but
it doesn't work, which means that the grass with msys shortcuts won't work
either if installed in a path with spaces.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/629#comment:7&gt;
GRASS GIS <http://grass.osgeo.org>

9

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by hamish):

Replying to [comment:7 cnielsen]:
> I can't think of a way the "dir /x" fix could be used within
> the installer to adjust things to the 8.3 path names, but if
> someone else can I'll implement it.
>
> The empty grass dir is a mount point for the install
> directory

yeah I realized that after posting,

> and allows the grass with msys shortcuts to
> be executed with {{{msys.bat /grass/bin/grass64 -wxpython}}}
> etc.

I don't really understand that.

> I tried quotes in the fstab file but it doesn't work,
> which means that the grass with msys shortcuts won't work
> either if installed in a path with spaces.

it is the fstab file that needs the PROGRA~1 style because it uses
whitespace as the column deliminator and has no quoting ability.

Hamish

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/629#comment:8&gt;
GRASS GIS <http://grass.osgeo.org>

10

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by hamish):

see also #637.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:9&gt;
GRASS GIS <http://grass.osgeo.org>

11

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by hamish):

Hi,

attached is a patch against the latest CVS version of msys.bat which fixes
some of the quoting issues. You'll have to `rem` out the "abort if spaces
in pathname" if you want to allow that.

Colin re. fstab: as with %WD% in the patch, maybe you can use %~dps0 to
get the 8.3 shortname string somehow?
http://www.robvanderwoude.com/ntfor.php

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:10&gt;
GRASS GIS <http://grass.osgeo.org>

12

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by hamish):

re.
{{{
Bad file descriptor: Files/GRASS
}}}

source:grass/trunk/lib/db/dbmi_base/dbmscap.c@#L171

?

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:11&gt;
GRASS GIS <http://grass.osgeo.org>

13

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by cnielsen):

Replying to [comment:10 hamish]:
> attached is a patch against the latest CVS version of msys.bat which
fixes some of the quoting issues. You'll have to `rem` out the "abort if
spaces in pathname" if you want to allow that.
As written it seems to abort whether there are spaces in the pathname or
not, I just tried it in c:\program files\ and c:\programs\ with the same
result. I think I've fixed it, see attached file (I've just attached
msys.bat not a patch. I could figure out svn diff for the CVS of
msys.bat...).

There's two bugs, one was your typo (hamish), the other I think is a bug
in the original msys.bat. Anyway, it successfully converts the path to 8.3
format.

Replying to [comment:8 hamish]:
> Replying to [comment:7 cnielsen]:
> > and allows the grass with msys shortcuts to
> > be executed with {{{msys.bat /grass/bin/grass64 -wxpython}}}
> > etc.
>
> I don't really understand that.
Replying to [comment:10 hamish]:
> Colin re. fstab: as with %WD% in the patch, maybe you can use %~dps0 to
get the 8.3 shortname string somehow?
http://www.robvanderwoude.com/ntfor.php

fstab is only needed for the "Grass with msys" shortcuts to work. They
work by calling msys.bat with the argument /grass/bin/grass64 -wxpython
(see the attached msys.bat lines 97 & 98. To fix this for spaces we have
two options though I'm not sure which will work.

1) fstab is written by the installer script with the install path, but I
can't find a way to get the nsi script to convert the path with %~dps0
before writing the fstab file.

2) If we can get msys.bat to convert a windows path (from an argument)
from (eg):

{{{ c:\Program Files\Grass\bin\grass64 -wxpython}}}

to:

{{{ /c/Program\ Files/Grass/bin/grass64 -wxpython }}}

then we wouldn't need fstab at all. The shortcut would use the windows
style path, and msys.bat would convert it into readable format for
msys.bat line 98

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:12&gt;
GRASS GIS <http://grass.osgeo.org>

14

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by hamish):

Replying to [comment:13 hamish]:
> any link to Nsi script help? seems like a fairly developed
> language; might have a system{} command.

maybe "Exec" + some `FOR /F ... SET UNIXPATH=~dps0` style DOS magic?
http://nsis.sourceforge.net/Docs/Chapter4.html#4.9.1.2

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:14&gt;
GRASS GIS <http://grass.osgeo.org>

15

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by cnielsen):

Replying to [comment:14 hamish]:
> maybe "Exec" + some `FOR /F ... SET UNIXPATH=~dps0` style DOS magic?
> http://nsis.sourceforge.net/Docs/Chapter4.html#4.9.1.2

No luck so far, I managed to have the NSI script write a little batch
file, which in turn writes the 8.3 shortname to a system variable with
SETX. This value is then written to the fstab value, however it always
writes the previous value of the variable instead... annoying. Also it is
not really preferable to have a system environment variable permanently
written for this purpose...

> GRASS-Installer.nsi has code for that already?
True but msys still has problems with "/c/Program\
Files/GRASS/bin/grass64" as it strips out the \.

On another similar note, I had an error with v.digit failing to write to
the dbf because the GIS Database path had a space in it (which is the
default in the installer).

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:15&gt;
GRASS GIS <http://grass.osgeo.org>

16

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by marisn):

Replying to [comment:15 cnielsen]:
>
> On another similar note, I had an error with v.digit failing to write to
the dbf because the GIS Database path had a space in it (which is the
default in the installer).

I commited to 6.5 a hack that was sitting in my commit queue for some
time. Please test r38898

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:16&gt;
GRASS GIS <http://grass.osgeo.org>

17

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by jef):

In case someone is not aware:

we use
[http://trac.osgeo.org/qgis/browser/trunk/qgis/src/plugins/grass/qgsgrassmodule.cpp#L55
GetShortPathName()] in QGIS' GRASS plugin to eliminate blanks from (some)
filenames and paths on Windows.

And NSIS has [http://nsis.sourceforge.net/Docs/Chapter4.html#4.9.3.9
GetFullPathName /SHORT]

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/629#comment:17&gt;
GRASS GIS <http://grass.osgeo.org>

18

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by cnielsen):

Replying to [comment:17 jef]:
> In case someone is not aware:
>
> And NSIS has [http://nsis.sourceforge.net/Docs/Chapter4.html#4.9.3.9
GetFullPathName /SHORT]

I wasn't aware, thanks jef. Patch for GRASS-Installer.nsi attached.

Replying to [comment:16 marisn]:
>I commited to 6.5 a hack that was sitting in my commit queue for some
time. Please test r38898

I tried this out and got "cannot open driver dbf". Below are the relevant
errors I think.

{{{
D2/3: c_table_definition()
D3/3: c_create_bgcmd()
cat integer
D2/3: c_create_table() field = 1 key = cat cols = cat integer
D1/3: Vect_default_field_info(): map = test3 field = 1
D2/3: drv = dbf db = $GISDBASE/$LOCATION_NAME/$MAPSET/dbf/
D2/3: driver = dbf, database = $GISDBASE/$LOCATION_NAME/$MAPSET/dbf/
D2/3: opendir c:/Programs/GIS/OSGeo4W/apps/grass/grass-6.5.svn\driver\db\

Invalid argument: can't _spawnl
WARNING: Cannot open driver dbf
}}}

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/629#comment:18&gt;
GRASS GIS <http://grass.osgeo.org>

19

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by marisn):

Replying to [comment:18 cnielsen]:
>
> Replying to [comment:16 marisn]:
> >I commited to 6.5 a hack that was sitting in my commit queue for some
time. Please test r38898
>
> I tried this out and got "cannot open driver dbf". Below are the
relevant errors I think.
>
> {{{
> D2/3: c_table_definition()
> D3/3: c_create_bgcmd()
> cat integer
> D2/3: c_create_table() field = 1 key = cat cols = cat integer
> D1/3: Vect_default_field_info(): map = test3 field = 1
> D2/3: drv = dbf db = $GISDBASE/$LOCATION_NAME/$MAPSET/dbf/
> D2/3: driver = dbf, database = $GISDBASE/$LOCATION_NAME/$MAPSET/dbf/
> D2/3: opendir
c:/Programs/GIS/OSGeo4W/apps/grass/grass-6.5.svn\driver\db\
>
> Invalid argument: can't _spawnl
> WARNING: Cannot open driver dbf
> }}}
Please apply patch I attached to this ticked and report results. It will
not fix issue, but hopefully provide aditional information.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/629#comment:19&gt;
GRASS GIS <http://grass.osgeo.org>

20

#629: WinGRASS: spaces in pathnames
---------------------------+------------------------------------------------
  Reporter: hamish | Owner: grass-dev@lists.osgeo.org
      Type: defect | Status: new
  Priority: normal | Milestone: 6.4.0
Component: Installation | Version: 6.4.0 RCs
Resolution: | Keywords: wingrass, msys
  Platform: MSWindows XP | Cpu: x86-32
---------------------------+------------------------------------------------
Comment (by cnielsen):

Replying to [comment:19 marisn]:
> Please apply patch I attached to this ticked and report results. It will
not fix issue, but hopefully provide aditional information.

The patch failed to compile

{{{
start.c: In function `db_start_driver':
start.c:224: error: too many arguments to function `db_syserror'
make: *** [OBJ.i686-pc-mingw32/start.o] Error 1
}}}

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/629#comment:20&gt;
GRASS GIS <http://grass.osgeo.org>