[GRASS-dev] python and GRASS_MESSAGE_FORMAT

GRASS GIS GRASS GIS developer
1

Hallo,

I would need to catch progress of r.los in one python script. I used
following approach

os.putenv("GRASS_MESSAGE_FORMAT","gui")
for line in os.popen("r.los in=elevation.dem out=tmp 2>&1").readlines():
    print line

It works well, but all the "lines" are printed out, just after r.los is
finished, but not while it is processing the raster map.

Could somebody help me, how to do it? Could you point me to tcl code,
which must solve this problem too?

Thank you

Jachym
--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://www.les-ejk.cz/pgp/jachym_cepicky-gpg.pub
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514

2

Jachym Cepicky wrote:

I would need to catch progress of r.los in one python script. I used
following approach

os.putenv("GRASS_MESSAGE_FORMAT","gui")
for line in os.popen("r.los in=elevation.dem out=tmp 2>&1").readlines():
    print line

It works well, but all the "lines" are printed out, just after r.los is
finished, but not while it is processing the raster map.

The documentation for the readlines() method says:

readlines([sizehint])

    Read until EOF using readline() and return a list containing the
    lines thus read. If the optional sizehint argument is present,
    instead of reading up to EOF, whole lines totalling approximately
    sizehint bytes (possibly after rounding up to an internal buffer
    size) are read. Objects implementing a file-like interface may
    choose to ignore sizehint if it cannot be implemented, or cannot
    be implemented efficiently.

The "until EOF" means that it isn't going to return until the command
has completed.

Could somebody help me, how to do it? Could you point me to tcl code,
which must solve this problem too?

Looking at the Python docs, you probably want something like:

  f = os.popen("r.los in=elevation.dem out=tmp 2>&1")
  try:
      for line in f:
          print line
  finally:
      f.close()

--
Glynn Clements <glynn@gclements.plus.com>

3

On Sat, Nov 25, 2006 at 06:56:14PM +0000, Glynn Clements wrote:

The documentation for the readlines() method says:

readlines([sizehint])

    Read until EOF using readline() and return a list containing the
    lines thus read. If the optional sizehint argument is present,
    instead of reading up to EOF, whole lines totalling approximately
    sizehint bytes (possibly after rounding up to an internal buffer
    size) are read. Objects implementing a file-like interface may
    choose to ignore sizehint if it cannot be implemented, or cannot
    be implemented efficiently.

The "until EOF" means that it isn't going to return until the command
has completed.

> Could somebody help me, how to do it? Could you point me to tcl code,
> which must solve this problem too?

Looking at the Python docs, you probably want something like:

  f = os.popen("r.los in=elevation.dem out=tmp 2>&1")
  try:
      for line in f:
          print line
  finally:
      f.close()

thanks, Glynn, but it does not seem to work - or it works the same way,
like readlines() does.

I have in the script:

import os
os.putenv("GRASS_MESSAGE_FORMAT","gui")

f = os.popen("r.los --o in=elevation.dem out=tmp \
        coordinate=603067.6875,4925297.0625 max_dist=10000 2>&1")
try:
    for line in f:
        print line
finally:
    f.close()

result is, lines are printed. but _after_ r.los finished it's
calcualtions..

do I understand something wrong?

thanks

jachym

--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://www.les-ejk.cz/pgp/jachym_cepicky-gpg.pub
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514

4

Hi Jachym. This looks like a buffered vs. unbuffered stream problem.

Looking at lib/gis/percent.c, the output of (the G_percent calls of)
r.los should flush at every write so if you try to popen in unbuffered
mode, you should get what you need: try os.popen("your_cmd","r",0).

HTH, Daniel.

On 11/25/06, Jachym Cepicky <jachym.cepicky@centrum.cz> wrote:

On Sat, Nov 25, 2006 at 06:56:14PM +0000, Glynn Clements wrote:
> The documentation for the readlines() method says:
>
> readlines([sizehint])
>
> Read until EOF using readline() and return a list containing the
> lines thus read. If the optional sizehint argument is present,
> instead of reading up to EOF, whole lines totalling approximately
> sizehint bytes (possibly after rounding up to an internal buffer
> size) are read. Objects implementing a file-like interface may
> choose to ignore sizehint if it cannot be implemented, or cannot
> be implemented efficiently.
>
> The "until EOF" means that it isn't going to return until the command
> has completed.
>
> > Could somebody help me, how to do it? Could you point me to tcl code,
> > which must solve this problem too?
>
> Looking at the Python docs, you probably want something like:
>
> f = os.popen("r.los in=elevation.dem out=tmp 2>&1")
> try:
> for line in f:
> print line
> finally:
> f.close()

thanks, Glynn, but it does not seem to work - or it works the same way,
like readlines() does.

I have in the script:

import os
os.putenv("GRASS_MESSAGE_FORMAT","gui")

f = os.popen("r.los --o in=elevation.dem out=tmp \
        coordinate=603067.6875,4925297.0625 max_dist=10000 2>&1")
try:
    for line in f:
        print line
finally:
    f.close()

result is, lines are printed. but _after_ r.los finished it's
calcualtions..

do I understand something wrong?

thanks

jachym

--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://www.les-ejk.cz/pgp/jachym_cepicky-gpg.pub
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFFaJyXyKt0uAjU4I8RAmlTAKDUnIxJiatd2h/kU/OGv/LI/I1NSQCgn9BJ
1yFsraD4e8VY3nEZeozW3JM=
=K8gc
-----END PGP SIGNATURE-----

_______________________________________________
grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

--
-- Daniel Calvelo Aros

5

On Sat, Nov 25, 2006 at 09:02:38PM -0500, Daniel Calvelo wrote:

Hi Jachym. This looks like a buffered vs. unbuffered stream problem.

Looking at lib/gis/percent.c, the output of (the G_percent calls of)
r.los should flush at every write so if you try to popen in unbuffered
mode, you should get what you need: try os.popen("your_cmd","r",0).

HTH, Daniel.

thanks for this hint, but it does not seem, that it would help.
behaviour remain same:

import os
os.putenv("GRASS_MESSAGE_FORMAT","gui")

f = os.popen("r.los --o in=elevation.dem out=tmp \
        coordinate=603067.6875,4925297.0625 max_dist=10000 2>&1","r",0)
try:
    for line in f:
        print line
finally:
    f.close()

jachym

On 11/25/06, Jachym Cepicky <jachym.cepicky@centrum.cz> wrote:
>On Sat, Nov 25, 2006 at 06:56:14PM +0000, Glynn Clements wrote:
>> The documentation for the readlines() method says:
>>
>> readlines([sizehint])
>>
>> Read until EOF using readline() and return a list containing the
>> lines thus read. If the optional sizehint argument is present,
>> instead of reading up to EOF, whole lines totalling approximately
>> sizehint bytes (possibly after rounding up to an internal buffer
>> size) are read. Objects implementing a file-like interface may
>> choose to ignore sizehint if it cannot be implemented, or cannot
>> be implemented efficiently.
>>
>> The "until EOF" means that it isn't going to return until the command
>> has completed.
>>
>> > Could somebody help me, how to do it? Could you point me to tcl code,
>> > which must solve this problem too?
>>
>> Looking at the Python docs, you probably want something like:
>>
>> f = os.popen("r.los in=elevation.dem out=tmp 2>&1")
>> try:
>> for line in f:
>> print line
>> finally:
>> f.close()
>
>thanks, Glynn, but it does not seem to work - or it works the same way,
>like readlines() does.
>
>I have in the script:
>
>import os
>os.putenv("GRASS_MESSAGE_FORMAT","gui")
>
>f = os.popen("r.los --o in=elevation.dem out=tmp \
> coordinate=603067.6875,4925297.0625 max_dist=10000 2>&1")
>try:
> for line in f:
> print line
>finally:
> f.close()
>
>result is, lines are printed. but _after_ r.los finished it's
>calcualtions..
>
>do I understand something wrong?
>
>thanks
>
>jachym
>
>--
>Jachym Cepicky
>e-mail: jachym.cepicky@centrum.cz
>URL: http://les-ejk.cz
>GPG: http://www.les-ejk.cz/pgp/jachym_cepicky-gpg.pub
>-----------------------------------------
>OFFICE:
>Department of Geoinformation Technologies
>Zemedelska 3
>613 00, Brno
>Czech Republick
>e-mail: xcepicky@node.mendelu.cz
>URL: http://mapserver.mendelu.cz
>Tel.: +420 545 134 514
>
>
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.4.3 (GNU/Linux)
>
>iD8DBQFFaJyXyKt0uAjU4I8RAmlTAKDUnIxJiatd2h/kU/OGv/LI/I1NSQCgn9BJ
>1yFsraD4e8VY3nEZeozW3JM=
>=K8gc
>-----END PGP SIGNATURE-----
>
>
>_______________________________________________
>grass-dev mailing list
>grass-dev@grass.itc.it
>http://grass.itc.it/mailman/listinfo/grass-dev
>
>

--
-- Daniel Calvelo Aros

_______________________________________________
grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://www.les-ejk.cz/pgp/jachym_cepicky-gpg.pub
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514

6

Jachym Cepicky wrote:

I would need to catch progress of r.los in one python script. I used
following approach

just a note: (from distant memory)

r.los is a very inefficient module*, and its G_percent() status is
probably not linear and possibly somewhat wrong. Keep the search radius
small or you'll tie up your server's CPU for a long long time!

[*] for a replacement see r.cva from GEM/add-ons

Hamish

7

On 11/26/06, Jachym Cepicky <jachym.cepicky@centrum.cz> wrote:

thanks for this hint, but it does not seem, that it would help.
behaviour remain same:

import os
os.putenv("GRASS_MESSAGE_FORMAT","gui")

f = os.popen("r.los --o in=elevation.dem out=tmp \
        coordinate=603067.6875,4925297.0625 max_dist=10000 2>&1","r",0)

Try with popoen3, to get all your pipes separate:

o,i,f = os.popen3("r.los --o in=elevation.dem out=tmp \
        coordinate=603067.6875,4925297.0625 max_dist=10000", bufsize=0 )

Note the omitted redirection of stderr to stdout.

try:
    for line in f:
        print line
finally:
    f.close()

jachym

--
-- Daniel Calvelo Aros

8

On Mon, Nov 27, 2006 at 12:02:16AM -0500, Daniel Calvelo wrote:

Try with popoen3, to get all your pipes separate:

o,i,f = os.popen3("r.los --o in=elevation.dem out=tmp \
       coordinate=603067.6875,4925297.0625 max_dist=10000", bufsize=0 )

Note the omitted redirection of stderr to stdout.

same result :-/

this should be solved, before we continue on newgeneration gui too :-/

thanks

jachym

>try:
> for line in f:
> print line
>finally:
> f.close()
>
>jachym

--
-- Daniel Calvelo Aros

_______________________________________________
grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://www.les-ejk.cz/pgp/jachym_cepicky-gpg.pub
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514