[GRASS-dev] grass.info versus print

GRASS GIS GRASS GIS developer
1

Hi, I am working on a script in which I want to provide the output formatted in three columns. I can use

nlength = 20
RES = [MAPy, vif, sqrtvif]
print ‘{0[0]:{1}s} {0[1]:8.2f} {0[2]:8.2f}’.format(RES, nlength)

This prints nicely on the command line. However, on the command output tab of the GUI, the order of the printed lines is messed up.

If I use instead:

nlength = 20
RES = [MAPy, vif, sqrtvif]
grass.info(‘{0[0]:{1}s} {0[1]:8.2f} {0[2]:8.2f}’.format(RES, nlength))

It looks good on the command output tab, but it is printed without the fixed width.

Is there a way to make to print text in multiple columns both on the command line and the command output tab of the addon GUI?

Paulo

2

Hi, Paulo!

On Mon, Apr 11, 2016 at 10:04 AM, Paulo van Breugel <p.vanbreugel@gmail.com>
wrote:

I want to provide the output formatted in three columns. I can use

nlength = 20
RES = [MAPy, vif, sqrtvif]
print '{0[0]:{1}s} {0[1]:8.2f} {0[2]:8.2f}'.format(RES, nlength)

This prints nicely on the command line. However, on the command output tab
of the GUI, the order of the printed lines is messed up.

Mixing of order can happen in GUI when using both stderr and stdout. What
exactly is messed up? A minimal working example is probably needed here.

If I use instead:

nlength = 20
RES = [MAPy, vif, sqrtvif]
grass.info('{0[0]:{1}s} {0[1]:8.2f} {0[2]:8.2f}'.format(RES, nlength))

It looks good on the command output tab, but it is printed without the
fixed width.

sys.stdout/print is the way. grass.script.info and others are for messages,
not output, and they go to stderr which might cause mixing of order with
stdout in GUI and I unfortunately don't know how to avoid it.

Vaclav

3

Hi Vaclav,

Thanks for the help. The problem of mixed order of output was indeed because I was mixing print (stdout) and grass.info (stderr). The code is here: . Following your pointer, I now replaced everything with print (I did not uploaded it yet) and now the columns display correctly on the console and in the command output tab of the addon GUI. Minor issue is that in the command output tab, empty lines are not printed. The script runs a loop, each time printing out a line with output. What I noticed is that if I use print, in the command output tab the output is all printed at once, while in the console, the output is printed one line at a time. Not really a problem, but I liked that the output is printed line by line, as that provides a sense of progress. Cheers, Paulo

···

On 11-04-16 17:18, Vaclav Petras wrote:

Hi, Paulo!

On Mon, Apr 11, 2016 at 10:04 AM, Paulo van Breugel <p.vanbreugel@gmail.com> wrote:

I want to provide the output formatted in three columns. I can use

nlength = 20
RES = [MAPy, vif, sqrtvif]
print ‘{0[0]:{1}s} {0[1]:8.2f} {0[2]:8.2f}’.format(RES, nlength)

This prints nicely on the command line. However, on the command output tab of the GUI, the order of the printed lines is messed up.

Mixing of order can happen in GUI when using both stderr and stdout. What exactly is messed up? A minimal working example is probably needed here.

https://trac.osgeo.org/grass/browser/grass-addons/grass7/raster/r.vif/

If I use instead:

nlength = 20
RES = [MAPy, vif, sqrtvif]
grass.info(‘{0[0]:{1}s} {0[1]:8.2f} {0[2]:8.2f}’.format(RES, nlength))

It looks good on the command output tab, but it is printed without the fixed width.

sys.stdout/print is the way. grass.script.info and others are for messages, not output, and they go to stderr which might cause mixing of order with stdout in GUI and I unfortunately don’t know how to avoid it.

4

Vaclav Petras wrote:

sys.stdout/print is the way. grass.script.info and others are for messages,
not output, and they go to stderr which might cause mixing of order with
stdout in GUI and I unfortunately don't know how to avoid it.

grass.script.info() executes "g.message -i", which calls
G_important_message(), which writes to stderr.

In general, scripts should use the g.message wrappers rather than
Python's "print" statement or os.write(), as that will honour any
configuration options (GRASS_VERBOSE, GRASS_MESSAGE_FORMAT,
--quiet/--verbose, etc).

--
Glynn Clements <glynn@gclements.plus.com>

5

On Thu, Apr 14, 2016 at 1:04 PM, Glynn Clements <glynn@gclements.plus.com>
wrote:

Vaclav Petras wrote:

> sys.stdout/print is the way. grass.script.info and others are for
messages,
> not output, and they go to stderr which might cause mixing of order with
> stdout in GUI and I unfortunately don't know how to avoid it.

grass.script.info() executes "g.message -i", which calls
G_important_message(), which writes to stderr.

In general, scripts should use the g.message wrappers rather than
Python's "print" statement or os.write(), as that will honour any
configuration options (GRASS_VERBOSE, GRASS_MESSAGE_FORMAT,
--quiet/--verbose, etc).

Thanks Glynn. It seems there are different opinions, see earlier comment
from Vaclav. Also in the examples on the Wiki (e.g., on
https://grasswiki.osgeo.org/wiki/GRASS_and_Python) and various addons I
have looked at, I can see both being used. Perhaps the most important thing
is that one should avoid mixing the two, which is something that maybe
could go somewhere in the wiki?

In my script (r.vif), both approaches have an issue with how output is
displayed. Using grass.info() results in columns not being printed with
fixed width in the command output, while they are in the console. On the
other hand, when using print() empty lines are not printed. I personally am
not too much bothered by both, so if you think I should use grass.info() I
am fine with that.

--
Glynn Clements <glynn@gclements.plus.com>

6

On Thu, Apr 14, 2016 at 7:33 AM, Paulo van Breugel <p.vanbreugel@gmail.com>
wrote:

On Thu, Apr 14, 2016 at 1:04 PM, Glynn Clements <glynn@gclements.plus.com>
wrote:

In general, scripts should use the g.message wrappers rather than
Python's "print" statement or os.write(), as that will honour any
configuration options (GRASS_VERBOSE, GRASS_MESSAGE_FORMAT,
--quiet/--verbose, etc).

Thanks Glynn. It seems there are different opinions, see earlier comment
from Vaclav.

Perhaps I was not completely clear before. I agree with Glynn.
grass.script.message() etc. for messages. print()/sys.stdout for text
output (e.g. tables).

Also in the examples on the Wiki (e.g., on
https://grasswiki.osgeo.org/wiki/GRASS_and_Python) and various addons I
have looked at, I can see both being used.

I would have to go through the individual examples but sometimes it might
be be clear what is a message to inform user about something and what is a
text output of the module. A rule of thumb can be that modules which output
maps, (usually) don't do text output, so they shouldn't use print().

Also for you personal scripts, you are free to use any solution you want.
That might be what the examples are showing. However, Python scripts which
are modules, for example in Addons, should adhere to the submitting
guidelines:

https://trac.osgeo.org/grass/wiki/Submitting/Python#Messages

Perhaps the most important thing is that one should avoid mixing the two,
which is something that maybe could go somewhere in the wiki?

Feel free to extent either wiki or guidelines if you think it is not clear
enough.

In my script (r.vif), both approaches have an issue with how output is
displayed. Using grass.info() results in columns not being printed with
fixed width in the command output, while they are in the console. On the
other hand, when using print() empty lines are not printed. I personally am
not too much bothered by both, so if you think I should use grass.info()
I am fine with that.

First, always use the above guidelines to decide if print() or one of the
g.message wrappers is the right solution. Second, look at the behavior in
the command line (that's the canonical one). The GUI might get things wrong
(and that might be bug).

Please, let me know what you think about it now,
Vaclav

7

In this case (r.vif) the output is not a map, but a list of values, i.e., text output (see below). Based on the guidelines, I would assume I should use print()? I wouldn’t mind doing so, but I probably should understand better myself what exactly the issue is with using both print() and grass.info() in one script. Based on what I read in the guidelines, I am not clear whether using both is against the guidelines as such, but technically it seems it can cause problems as you explained (and I experienced). OK, that is good to know

···

On 14-04-16 14:27, Vaclav Petras wrote:

On Thu, Apr 14, 2016 at 7:33 AM, Paulo van Breugel <p.vanbreugel@gmail.com> wrote:

On Thu, Apr 14, 2016 at 1:04 PM, Glynn Clements <glynn@gclements.plus.com> wrote:

In general, scripts should use the g.message wrappers rather than
Python’s “print” statement or os.write(), as that will honour any
configuration options (GRASS_VERBOSE, GRASS_MESSAGE_FORMAT,
–quiet/–verbose, etc).

Thanks Glynn. It seems there are different opinions, see earlier comment from Vaclav.

Perhaps I was not completely clear before. I agree with Glynn. grass.script.message() etc. for messages. print()/sys.stdout for text output (e.g. tables).

Also in the examples on the Wiki (e.g., on https://grasswiki.osgeo.org/wiki/GRASS_and_Python) and various addons I have looked at, I can see both being used.

I would have to go through the individual examples but sometimes it might be be clear what is a message to inform user about something and what is a text output of the module. A rule of thumb can be that modules which output maps, (usually) don’t do text output, so they shouldn’t use print().

All variables

----------------------------------------

variable                           vif      sqrtvif

2000_01_precip@climate_1970_2012       2.30     1.52

2000_03_tempmean@climate_1970_2012    24.95     5.00

2000_11_precip@climate_1970_2012       3.54     1.88

2001_02_precip@climate_1970_2012       4.55     2.13

2001_04_tempmean@climate_1970_2012    25.01     5.00

Variable '2001_04_tempmean@climate_1970_2012' removed

----------------------------------------

variable                           vif      sqrtvif

2000_01_precip@climate_1970_2012       2.30     1.52

2000_03_tempmean@climate_1970_2012     2.19     1.48

2000_11_precip@climate_1970_2012       3.48     1.87

2001_02_precip@climate_1970_2012       4.54     2.13

Final selected variables are: 2000_01_precip, 2000_03_tempmean, 2000_11_precip, 2001_02_precip

`with as maximum VIF: 5.69787583189`

Also for you personal scripts, you are free to use any solution you want. That might be what the examples are showing. However, Python scripts which are modules, for example in Addons, should adhere to the submitting guidelines:

https://trac.osgeo.org/grass/wiki/Submitting/Python#Messages

Perhaps the most important thing is that one should avoid mixing the two, which is something that maybe could go somewhere in the wiki?

Feel free to extent either wiki or guidelines if you think it is not clear enough.

In my script (r.vif), both approaches have an issue with how output is displayed. Using grass.info() results in columns not being printed with fixed width in the command output, while they are in the console. On the other hand, when using print() empty lines are not printed. I personally am not too much bothered by both, so if you think I should use grass.info() I am fine with that.

First, always use the above guidelines to decide if print() or one of the g.message wrappers is the right solution. Second, look at the behavior in the command line (that’s the canonical one). The GUI might get things wrong (and that might be bug).

Please, let me know what you think about it now,

Vaclav

8

On Thu, Apr 14, 2016 at 9:55 AM, Paulo van Breugel <p.vanbreugel@gmail.com>
wrote:

I would have to go through the individual examples but sometimes it might
be be clear what is a message to inform user about something and what is a
text output of the module. A rule of thumb can be that modules which output
maps, (usually) don't do text output, so they shouldn't use print().

In this case (r.vif) the output is not a map, but a list of values, i.e.,
text output (see below). Based on the guidelines, I would assume I should
use print()?

Then print() is the right thing. I would recommend finding a similar module
and see its behavior. For example, r.univar will give you human readable
report and also parseable output (in this case by
grass.script.parse_command()). Both are written to standard output (the
g.message wrappers use standard error output).

All variables

----------------------------------------

variable vif sqrtvif

2000_01_precip@climate_1970_2012 2.30 1.52

2000_03_tempmean@climate_1970_2012 24.95 5.00

2000_11_precip@climate_1970_2012 3.54 1.88

2001_02_precip@climate_1970_2012 4.55 2.13

2001_04_tempmean@climate_1970_2012 25.01 5.00

Variable '2001_04_tempmean@climate_1970_2012' removed

----------------------------------------

variable vif sqrtvif

2000_01_precip@climate_1970_2012 2.30 1.52

2000_03_tempmean@climate_1970_2012 2.19 1.48

2000_11_precip@climate_1970_2012 3.48 1.87

2001_02_precip@climate_1970_2012 4.54 2.13

Final selected variables are: 2000_01_precip, 2000_03_tempmean, 2000_11_precip, 2001_02_precip

with as maximum VIF: 5.69787583189

This definitively looks like something from r.report or r.info.

Perhaps the most important thing is that one should avoid mixing the two,

which is something that maybe could go somewhere in the wiki?

Feel free to extent either wiki or guidelines if you think it is not clear
enough.

I wouldn't mind doing so, but I probably should understand better myself
what exactly the issue is with using both print() and grass.info() in one
script. Based on what I read in the guidelines, I am not clear whether
using both is against the guidelines as such,

You can use both in one script. The difference is what you use them for.
For example, a lot of modules will tell which map was created (this is done
through g.message or its C equivalents), while g.region -p will use
standard output. Some cases are unclear, for example v.in.lidar uses
standard output (fprintf(stdout,...)) to tell how many points imported.
G_message() might be more appropriate here because it is an message to the
user while the output is the vector map. If G_message() would be used, this
output could be silenced by --quiet.

but technically it seems it can cause problems as you explained (and I
experienced).

They are mostly related to syncing of stdout and strerr streams which is
more challenging for the GUI (and may need some fixing there).

9

OK, will do that

···

On 14-04-16 16:20, Vaclav Petras wrote:

On Thu, Apr 14, 2016 at 9:55 AM, Paulo van Breugel <p.vanbreugel@gmail.com> wrote:

I would have to go through the individual examples but sometimes it might be be clear what is a message to inform user about something and what is a text output of the module. A rule of thumb can be that modules which output maps, (usually) don’t do text output, so they shouldn’t use print().

In this case (r.vif) the output is not a map, but a list of values, i.e., text output (see below). Based on the guidelines, I would assume I should use print()?

Then print() is the right thing. I would recommend finding a similar module and see its behavior. For example, r.univar will give you human readable report and also parseable output (in this case by grass.script.parse_command()). Both are written to standard output (the g.message wrappers use standard error output).

All variables

----------------------------------------

variable                           vif      sqrtvif

2000_01_precip@climate_1970_2012       2.30     1.52

2000_03_tempmean@climate_1970_2012    24.95     5.00

2000_11_precip@climate_1970_2012       3.54     1.88

2001_02_precip@climate_1970_2012       4.55     2.13

2001_04_tempmean@climate_1970_2012    25.01     5.00

Variable '2001_04_tempmean@climate_1970_2012' removed

----------------------------------------

variable                           vif      sqrtvif

2000_01_precip@climate_1970_2012       2.30     1.52

2000_03_tempmean@climate_1970_2012     2.19     1.48

2000_11_precip@climate_1970_2012       3.48     1.87

2001_02_precip@climate_1970_2012       4.54     2.13

Final selected variables are: 2000_01_precip, 2000_03_tempmean, 2000_11_precip, 2001_02_precip

`with as maximum VIF: 5.69787583189`

This definitively looks like something from r.report or r.info.

Perhaps the most important thing is that one should avoid mixing the two, which is something that maybe could go somewhere in the wiki?

Feel free to extent either wiki or guidelines if you think it is not clear enough.

I wouldn’t mind doing so, but I probably should understand better myself what exactly the issue is with using both print() and grass.info() in one script. Based on what I read in the guidelines, I am not clear whether using both is against the guidelines as such,

You can use both in one script. The difference is what you use them for. For example, a lot of modules will tell which map was created (this is done through g.message or its C equivalents), while g.region -p will use standard output. Some cases are unclear, for example v.in.lidar uses standard output (fprintf(stdout,…)) to tell how many points imported. G_message() might be more appropriate here because it is an message to the user while the output is the vector map. If G_message() would be used, this output could be silenced by --quiet.

but technically it seems it can cause problems as you explained (and I experienced).

They are mostly related to syncing of stdout and strerr streams which is more challenging for the GUI (and may need some fixing there).

10

Vaclav Petras wrote:

Perhaps I was not completely clear before. I agree with Glynn.
grass.script.message() etc. for messages. print()/sys.stdout for text
output (e.g. tables).

In short: g.message for "messages", print or sys.stdout.write() for
"data".

The reason why Unix has stdout and stderr is so that you can redirect
stdout to a file which having stderr associated with the terminal. So
messages intended for the user don't interfere with parsing the data,
and redirecting the data to the file doesn't hide messages from the
user.

So, assuming that the two streams will be separated should provide
some clues as to which is appropriate for any given case.

--
Glynn Clements <glynn@gclements.plus.com>