[GRASSLIST:787] bash scripts don't wait for grass

Hello,

Maybe unrelated but I remember a similar question on the list.

When I run a ~400 lines bash script to batch process more than just a couple of files, the script kind of of gets tangled and stops before processing the last file (it is basically v.in.ascii importing plus v.rast.idw interpolating, and some Perl scripts), as if some overlapping between child processes occurred.

I tried with the "wait" bash command which should have fixed it, but it still occurs. Is it a known issue? Or is it related to the fact that I am running grass60 on cygwin?

Any hints to make grass batch processing more reliable will be appreciated.

Thanks,

Luigi

Luigi Ponti wrote:

Hello,

Maybe unrelated but I remember a similar question on the list.

When I run a ~400 lines bash script to batch process more than just a couple of files, the script kind of of gets tangled and stops before processing the last file (it is basically v.in.ascii importing plus v.rast.idw interpolating, and some Perl scripts), as if some overlapping between child processes occurred.

I tried with the "wait" bash command which should have fixed it, but it still occurs. Is it a known issue? Or is it related to the fact that I am running grass60 on cygwin?

Any hints to make grass batch processing more reliable will be appreciated.

Thanks,

Luigi

--
Benjamin Ducke, M.A.
Archäoinformatik
(Archaeo-Information Science)
Institut für Ur- und Frühgeschichte
(Institute of Archaeology)
Christian-Albrechts-Universität zu Kiel
Johanna-Mestorf-Straße 2-6
D 24098 Kiel
Germany

Tel.: ++49 (0)431 880-3378 / -3379
Fax : ++49 (0)431 880-7300
www.uni-kiel.de/ufg

Hmmm, I think there is one major problem with CygWin and
Unix scripting in general: CygWin really is Windows, thus
incapable of doing a few basic things that true Unix can.

One really sore spot is the fact that Windows cannot redirect
the stderr output. The CygWin console suffers from the same
defficiency. If you have a module that tries to redirect
its stderr to a file, e.g. for hiding status messages, you
will get a total mess!

I ran into the same problem a while ago and found no solution
other than this:

Disable all sorts of message hiding, quiet options etc. in
the modules causing problems and make them dump there full
verbose output to the console. This is the only place where
stderr does not make a mess in Windows.
This means that you will get a lot of unwanted status messages
on your console, but at least your processes will run in clean
order.

If anyone has a better solution for this, I would be very
interested to know, indeed!

Best,

Benjamin

Luigi Ponti wrote:

Hello,

Maybe unrelated but I remember a similar question on the list.

When I run a ~400 lines bash script to batch process more than just a couple of files, the script kind of of gets tangled and stops before processing the last file (it is basically v.in.ascii importing plus v.rast.idw interpolating, and some Perl scripts), as if some overlapping between child processes occurred.

I tried with the "wait" bash command which should have fixed it, but it still occurs. Is it a known issue? Or is it related to the fact that I am running grass60 on cygwin?

Any hints to make grass batch processing more reliable will be appreciated.

Thanks,

Luigi

--
Benjamin Ducke, M.A.
Archäoinformatik
(Archaeo-Information Science)
Institut für Ur- und Frühgeschichte
(Institute of Archaeology)
Christian-Albrechts-Universität zu Kiel
Johanna-Mestorf-Straße 2-6
D 24098 Kiel
Germany

Tel.: ++49 (0)431 880-3378 / -3379
Fax : ++49 (0)431 880-7300
www.uni-kiel.de/ufg

One really sore spot is the fact that Windows cannot redirect
the stderr output. The CygWin console suffers from the same
defficiency. If you have a module that tries to redirect
its stderr to a file, e.g. for hiding status messages, you
will get a total mess!

I ran into the same problem a while ago and found no solution
other than this:

Disable all sorts of message hiding, quiet options etc. in
the modules causing problems and make them dump there full
verbose output to the console. This is the only place where
stderr does not make a mess in Windows.

fyi, for messages coming from G_warning() have a look at
G_suppress_warnings() and G_set_error_routine() in lib/gis/error.c

Hamish

Thanks Benjamin and Radim for the reply.

I will try to make the script as verbose as I can.
Another thing I find useful, although not exactly elegant, is to send the script to "sleep" for a couple of seconds --you have to adjust it iteratively. It is not a path to efficiency but if nothing else works, you can at least get the job done.

Ciao,

Luigi

Benjamin Ducke wrote:

Hmmm, I think there is one major problem with CygWin and
Unix scripting in general: CygWin really is Windows, thus
incapable of doing a few basic things that true Unix can.

One really sore spot is the fact that Windows cannot redirect
the stderr output. The CygWin console suffers from the same
defficiency. If you have a module that tries to redirect
its stderr to a file, e.g. for hiding status messages, you
will get a total mess!

I ran into the same problem a while ago and found no solution
other than this:

Disable all sorts of message hiding, quiet options etc. in
the modules causing problems and make them dump there full
verbose output to the console. This is the only place where
stderr does not make a mess in Windows.
This means that you will get a lot of unwanted status messages
on your console, but at least your processes will run in clean
order.

If anyone has a better solution for this, I would be very
interested to know, indeed!

Best,

Benjamin

Luigi Ponti wrote:

Hello,

Maybe unrelated but I remember a similar question on the list.

When I run a ~400 lines bash script to batch process more than just a couple of files, the script kind of of gets tangled and stops before processing the last file (it is basically v.in.ascii importing plus v.rast.idw interpolating, and some Perl scripts), as if some overlapping between child processes occurred.

I tried with the "wait" bash command which should have fixed it, but it still occurs. Is it a known issue? Or is it related to the fact that I am running grass60 on cygwin?

Any hints to make grass batch processing more reliable will be appreciated.

Thanks,

Luigi

Hello Benjamin,

I am trying to have GRASS run a script at startup. I am doing this using your grass-remote script which, if I understand correctly, takes location path and script name as arguments. Since I am working on a cygwin/grass60 system, I would start grass-remote with a windows batch file (basically the one that ships with the cygwin binaries of grass). However, it does not work properly because programs like clean_temp.exe can't find libraries such as libgrass_gis.6.0.0.dll

In the mailing list archive, I found a reply by Glynn Clements and Thomas Adams
http://grass.itc.it/pipermail/grassuser/2005-September/030495.html
saying that a script needs to set up the necessary variables (e.g. GISBASE, GISRC) itself before attempting to run GRASS commands. Since grass-remote is shaped on Init.sh, it should set necessary variables.

Do you think this task is doable on cygwin/grass60? What am I missing?

Thanks in advance,

Luigi