[GRASS-dev] [GRASS-user] why is v.build.all (and many others) a windows batch file and not an executable?

GRASS GIS GRASS GIS developer
1

(moved temporarily here for catching expert answers, then I'll
transfer the result to grass-user to avoid cross-posting)

The point is that the user wants to run without extra hassle
(parameter updates are ok of course) existing G6 scripts in a G7
session on Windows:

------- start fwd -------
On Sat, Apr 30, 2016 at 10:53 AM, Bartolomei.Chris wrote:

Ok (I guess) but this causes severe problems running shell scripts that call out the GRASS modules ... there is no way of knowing which modules are compiled executables (which run fine from the shell script) and which ones are Windows Batch files (which DON'T run when called from a script) ... unless you run the script and figure out which modules make it crash. If you look in the bin directory, you'll see a mix of both ... for example r.mask, v.build.all and v.db.addcolumn are Windows Batch files which you can''t run from a shell script - you have to add the "python {path to script}/{script}.py" to run it whereas in the same bin directory v.build, v.clean, r.what, etc are compiled executables.
This is really really (really) bad. There are a lot of legacy shell scripts that will need extensive rewriting for which there are no resources to do. It's bad enough that msys is gone, the topology is no longer valid (have to update all data), a large amount of the module syntax options have changed, and so on ... for me all of this grief is because v.net doesn't snap points to the network and the fix (in v7) won't be ported to v6.4.4 .... They modules should all be compiled executables so legacy shell scripts users have can run them ... please make at least ONE thing about this "upgrade" semi-straightforward. It's been a nightmare.

------- end fwd -------

I hope someone can give suggestions here.

thanks
Markus

2

Hi,

Seems a bit like an edge case with shell scripts on Windows...

Anyway, what about using a script to do the replacement?
Something like (did not test the details):

scripts=$(ls OSGeo4W/apps/grass/grass-7.0.3/scripts/*.py | sed 's/.\{4\}$//')
cp MY_OLD_GRASS_6_SCRIPT.sh MY_NEW_GRASS_7_SCRIPT.sh

for s in $scripts
do
cat MY_NEW_GRASS_7_SCRIPT.sh | sed "s|$s|python OSGeo4W/apps/grass/grass-7.0.3/scripts/$s.py|g" > MY_NEW_GRASS_7_SCRIPT.sh.tmp
mv MY_NEW_GRASS_7_SCRIPT.sh.tmp MY_NEW_GRASS_7_SCRIPT.sh
done

Cheers
Stefan

-----Original Message-----
From: grass-dev [mailto:grass-dev-bounces@lists.osgeo.org] On Behalf Of Markus Neteler
Sent: 30. april 2016 23:32
To: GRASS developers list <grass-dev@lists.osgeo.org>
Subject: Re: [GRASS-dev] [GRASS-user] why is v.build.all (and many others) a windows batch file and not an executable?

(moved temporarily here for catching expert answers, then I'll transfer the result to grass-user to avoid cross-posting)

The point is that the user wants to run without extra hassle (parameter updates are ok of course) existing G6 scripts in a G7 session on Windows:

------- start fwd -------
On Sat, Apr 30, 2016 at 10:53 AM, Bartolomei.Chris wrote:

Ok (I guess) but this causes severe problems running shell scripts that call out the GRASS modules ... there is no way of knowing which modules are compiled executables (which run fine from the shell script) and which ones are Windows Batch files (which DON'T run when called from a script) ... unless you run the script and figure out which modules make it crash. If you look in the bin directory, you'll see a mix of both ... for example r.mask, v.build.all and v.db.addcolumn are Windows Batch files which you can''t run from a shell script - you have to add the "python {path to script}/{script}.py" to run it whereas in the same bin directory v.build, v.clean, r.what, etc are compiled executables.
This is really really (really) bad. There are a lot of legacy shell scripts that will need extensive rewriting for which there are no resources to do. It's bad enough that msys is gone, the topology is no longer valid (have to update all data), a large amount of the module syntax options have changed, and so on ... for me all of this grief is because v.net doesn't snap points to the network and the fix (in v7) won't be ported to v6.4.4 .... They modules should all be compiled executables so legacy shell scripts users have can run them ... please make at least ONE thing about this "upgrade" semi-straightforward. It's been a nightmare.

------- end fwd -------

I hope someone can give suggestions here.

thanks
Markus
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

3

Markus Neteler wrote

(moved temporarily here for catching expert answers, then I'll
transfer the result to grass-user to avoid cross-posting)

The point is that the user wants to run without extra hassle
(parameter updates are ok of course) existing G6 scripts in a G7
session on Windows:

------- start fwd -------
On Sat, Apr 30, 2016 at 10:53 AM, Bartolomei.Chris wrote:

Ok (I guess) but this causes severe problems running shell scripts that
call out the GRASS modules ... there is no way of knowing which modules
are compiled executables (which run fine from the shell script) and which
ones are Windows Batch files (which DON'T run when called from a script)
... unless you run the script and figure out which modules make it crash.
If you look in the bin directory, you'll see a mix of both ... for
example r.mask, v.build.all and v.db.addcolumn are Windows Batch files
which you can''t run from a shell script - you have to add the "python
{path to script}/{script}.py" to run it whereas in the same bin directory
v.build, v.clean, r.what, etc are compiled executables.
This is really really (really) bad. There are a lot of legacy shell
scripts that will need extensive rewriting for which there are no
resources to do. It's bad enough that msys is gone, the topology is no
longer valid (have to update all data), a large amount of the module
syntax options have changed, and so on ... for me all of this grief is
because v.net doesn't snap points to the network and the fix (in v7)
won't be ported to v6.4.4 .... They modules should all be compiled
executables so legacy shell scripts users have can run them ... please
make at least ONE thing about this "upgrade" semi-straightforward. It's
been a nightmare.

------- end fwd -------

I hope someone can give suggestions here.

thanks
Markus
__________________________

I haven't a closer look at this issue.

AFAIU it may be a path issue: the path to the script folder isn't exported
to path.

in winGRASS 7 only the bin-folder is visible in %PATH%, as we put there the
wrapper bat-files, which invokes the script with the own shipped python
interpreter (=GRASS_PYTHON variable).

the reason for this wrapper bat files is to use the winGRASS shipped python
interpreter and not the system wide which may be incompatible.

for this issue with shell scripts in msys, it should be looked first what is
in %PATH%/$PATH and may be missing; and which variables (e.g.
GRASS_PYTHON,...) are set.

-----
best regards
Helmut
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Re-GRASS-user-why-is-v-build-all-and-many-others-a-windows-batch-file-and-not-an-executable-tp5263689p5263712.html
Sent from the Grass - Dev mailing list archive at Nabble.com.

4

Markus Neteler wrote

(moved temporarily here for catching expert answers, then I'll
transfer the result to grass-user to avoid cross-posting)

The point is that the user wants to run without extra hassle
(parameter updates are ok of course) existing G6 scripts in a G7
session on Windows:

------- start fwd -------
On Sat, Apr 30, 2016 at 10:53 AM, Bartolomei.Chris wrote:

Ok (I guess) but this causes severe problems running shell scripts that
call out the GRASS modules ... there is no way of knowing which modules
are compiled executables (which run fine from the shell script) and which
ones are Windows Batch files (which DON'T run when called from a script)
... unless you run the script and figure out which modules make it crash.
If you look in the bin directory, you'll see a mix of both ... for
example r.mask, v.build.all and v.db.addcolumn are Windows Batch files
which you can''t run from a shell script - you have to add the "python
{path to script}/{script}.py" to run it whereas in the same bin directory
v.build, v.clean, r.what, etc are compiled executables.
This is really really (really) bad. There are a lot of legacy shell
scripts that will need extensive rewriting for which there are no
resources to do. It's bad enough that msys is gone, the topology is no
longer valid (have to update all data), a large amount of the module
syntax options have changed, and so on ... for me all of this grief is
because v.net doesn't snap points to the network and the fix (in v7)
won't be ported to v6.4.4 .... They modules should all be compiled
executables so legacy shell scripts users have can run them ... please
make at least ONE thing about this "upgrade" semi-straightforward. It's
been a nightmare.

------- end fwd -------

I hope someone can give suggestions here.

thanks
Markus

in the MS windows environment executables are normally identified by file
extensions (e.g. exe, bat, cmd, ...) and can be invoked by typing only the
filename e.g. in a windows console.

there is a limitation of the msys environment that only files with the
exe-extension are recognized as executables.

a simple test in a msys environment with 2 files, which start in a windows
console by typing only the file name:

mytest.bat
echo test bat extension in msys

mytest.cmd
echo test cmd extension in msys

---------------------------------
$ cd "C:\tmp\testmsys"

/c/tmp/testmsys
$ ls
mytest.bat mytest.cmd

/c/tmp/testmsys
$ mytest
sh": mytest: command not found

/c/tmp/testmsys
$ mytest.bat
test bat extension in msys

/c/tmp/testmsys
$ mytest
sh": mytest: command not found

/c/tmp/testmsys
$ mytest.cmd
test cmd extension in msys
---------------------------------

and now the same test in a native windows console:

---------------------------------
C:\tmp\testmsys>dir /b
mytest.bat
mytest.cmd

C:\tmp\testmsys>mytest

C:\tmp\testmsys>echo test bat extension in msys
test bat extension in msys
---------------------------------

this clear limitation of the msys environments regarding executable file
extensions was one of the reasons to drop msys in winGRASS.

-----
best regards
Helmut
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Re-GRASS-user-why-is-v-build-all-and-many-others-a-windows-batch-file-and-not-an-executable-tp5263689p5263729.html
Sent from the Grass - Dev mailing list archive at Nabble.com.