[GRASS-dev] r.flow and G_message

GRASS GIS GRASS GIS developer
1

hallo,

i do not know, how to solve fprints in r.flow.

The module use it's own function to print messages (diag)

When i removed this function and rewrite everything with help of
G_message, the result is not well-arranged:

r.flow elevin=elevation.dem aspin=aspect barin=roads flout=tmp1
lgout=tmp2 dsout=tmp3 --o
Opening output files...
WARNING: The vector 'tmp1' already exists and will be overwritten.
Done
Allocating memory: elevation
aspect
barrier
density
flowline header
e/w distances
quantization tolerances
Reading input files: elevation...
aspect
barrier
Precomputing: tangents
e/w distances
quantization tolerances
re-oriented aspects
Working...
Done%
Writing density file...
Done
Closing files...
Building topology ...
0 primitives registered
0 areas built
0 isles built
Attaching islands:
Attaching centroids: Topology was built.
Number of nodes : 0
Number of primitives: 0
Number of points : 0
Number of lines : 0
Number of boundaries: 0
Number of centroids : 0
Number of areas : 0
Number of isles : 0
Done
De-allocating memory...
Done

What to do? Should I remove all informations about reading, precomputing
and allocating for each raster map? I suggest, that the most verbose
output could look like this:

Opening output files...
WARNING: The vector 'tmp1' already exists and will be overwritten.
Allocating memory for maps
Reading input files
Precomputing
Working
Writing density file
Closing files
Building topology
0 primitives registered
0 areas built
0 isles built
Attaching islands:
Attaching centroids: Topology was built.
Number of nodes : 0
Number of primitives: 0
Number of points : 0
Number of lines : 0
Number of boundaries: 0
Number of centroids : 0
Number of areas : 0
Number of isles : 0
Done
De-allocating memory

what would you say?

jachym
--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://les-ejk.cz/gnupg_public_key/jachym_cepicky-gpg_public_key.asc
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514

2

I warned the list about this.

Here are the options:
- Go back to fprintf() statements
- Rewrite the code so they are properly formatted with G_message(), etc.
- Add another function simiar to G_message() that does not automatically
append '\n' (this has already been shot down).

On Thu, 2006-10-19 at 21:21 +0200, Jachym Cepicky wrote:

hallo,

i do not know, how to solve fprints in r.flow.

The module use it's own function to print messages (diag)

When i removed this function and rewrite everything with help of
G_message, the result is not well-arranged:

r.flow elevin=elevation.dem aspin=aspect barin=roads flout=tmp1
lgout=tmp2 dsout=tmp3 --o
Opening output files...
WARNING: The vector 'tmp1' already exists and will be overwritten.
Done
Allocating memory: elevation
aspect
barrier
density
flowline header
e/w distances
quantization tolerances
Reading input files: elevation...
aspect
barrier
Precomputing: tangents
e/w distances
quantization tolerances
re-oriented aspects
Working...
Done%
Writing density file...
Done
Closing files...
Building topology ...
0 primitives registered
0 areas built
0 isles built
Attaching islands:
Attaching centroids: Topology was built.
Number of nodes : 0
Number of primitives: 0
Number of points : 0
Number of lines : 0
Number of boundaries: 0
Number of centroids : 0
Number of areas : 0
Number of isles : 0
Done
De-allocating memory...
Done

What to do? Should I remove all informations about reading, precomputing
and allocating for each raster map? I suggest, that the most verbose
output could look like this:

Opening output files...
WARNING: The vector 'tmp1' already exists and will be overwritten.
Allocating memory for maps
Reading input files
Precomputing
Working
Writing density file
Closing files
Building topology
0 primitives registered
0 areas built
0 isles built
Attaching islands:
Attaching centroids: Topology was built.
Number of nodes : 0
Number of primitives: 0
Number of points : 0
Number of lines : 0
Number of boundaries: 0
Number of centroids : 0
Number of areas : 0
Number of isles : 0
Done
De-allocating memory

what would you say?

--
Brad Douglas <rez touchofmadness com> KB8UYR/6
Address: 37.493,-121.924 / WGS84 National Map Corps #TNMC-3785

3

Jachym Cepicky wrote:

i do not know, how to solve fprints in r.flow.

The module use it's own function to print messages (diag)

When i removed this function and rewrite everything with help of
G_message, the result is not well-arranged:

r.flow elevin=elevation.dem aspin=aspect barin=roads flout=tmp1
lgout=tmp2 dsout=tmp3 --o
Opening output files...
WARNING: The vector 'tmp1' already exists and will be overwritten.
Done
Allocating memory: elevation
aspect
barrier
density
flowline header
e/w distances
quantization tolerances
Reading input files: elevation...
aspect
barrier

[snip]

What to do? Should I remove all informations about reading, precomputing
and allocating for each raster map? I suggest, that the most verbose
output could look like this:

In cases where the module currently outputs multiple distinct messages
on a single line, I would add the prefix to each call, so in the above
case you would get:

Allocating memory: elevation
Allocating memory: aspect
Allocating memory: barrier
Allocating memory: density
Allocating memory: flowline header
Allocating memory: e/w distances
Allocating memory: quantization tolerances
Reading input files: elevation
Reading input files: aspect
Reading input files: barrier

etc.

In this case, the "Allocating memory" messages should probably be
G_debug() rather than G_message(). The "Reading input files" messages
are more useful (and correspond to tasks which will typically take
longer), and so should use G_message().

[On that topic, I would suggest merging verbose/debug levels so that
higher settings of GRASS_VERBOSE enable debugging information, rather
than having a separate variable.]

Also, I would suggest an additional verbosity level between "message"
and "debug" which would be used for messages which are relatively
unimportant, yet still meant for users rather than developers, with
"debug" being reserved for information which is only useful for, well,
debugging.

FWIW, the Unix syslog mechanism has levels named emerg, alert, crit,
err, warning, notice, info, and debug. We don't distinguish between
different severities of error (all errors are fatal), so emerg, alert
and crit aren't particularly relevant, but the others roughly
correspond to:

  err G_fatal_error
  warning G_warning
  notice/info G_message
  debug G_debug

I suggest keeping G_message() for more useful messages ("notice")) and
adding a new G_{verbose,info,detail,<whatever>} function for the less
useful user-level messages ("info") generated by more verbose modules
such as r.flow.

Also, it may be worth making a distinction between progress
indications and other messages. Non-interactive environments (e.g. CGI
applications or cron jobs) don't really have any use for progress
information, even if you want a relatively high level of verbosity in
other regards (e.g. lots of information about the data being processed
and the results).

--
Glynn Clements <glynn@gclements.plus.com>

4

Brad Douglas wrote:

I warned the list about this.

Here are the options:

- Go back to fprintf() statements

- Rewrite the code so they are properly formatted with G_message(), etc.

- Add another function simiar to G_message() that does not automatically
append '\n' (this has already been shot down).

I prefer #2.

#1 results in yet another module which needs to be individually
modified whenever we change the conventions relating to how modules
communicate with their "user" (which may be either a human or a
program).

#3 isn't much use if stdout/stderr refer to a pipe which is being read
a line at a time (fgets() etc) from e.g. gis.m.

The approach currently used by r.flow is totally oriented towards a
terminal, and is something we should be moving away from, IMHO.

--
Glynn Clements <glynn@gclements.plus.com>

5

Jachym,

We're using r.flow heavily in a new landscape dynamics modeling package. The
information it prints to stderr has been a real problem all along for using
this from the GUI and in scripts. We've finally gotten around it, but at
times this almost made the module unusable.

I am strongly in favor of your suggestion to remove and/or reduce this extra
verbage that is a holdover from a time when all modules were run from the
command line by the user and without a GUI.

Thanks very much for looking into this.

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: Jachym Cepicky <jachym.cepicky@centrum.cz>
Date: Thu, 19 Oct 2006 21:21:04 +0200
To: <grass-dev@grass.itc.it>
Subject: [GRASS-dev] r.flow and G_message

hallo,

i do not know, how to solve fprints in r.flow.

The module use it's own function to print messages (diag)

When i removed this function and rewrite everything with help of
G_message, the result is not well-arranged:

r.flow elevin=elevation.dem aspin=aspect barin=roads flout=tmp1
lgout=tmp2 dsout=tmp3 --o
Opening output files...
WARNING: The vector 'tmp1' already exists and will be overwritten.
Done
Allocating memory: elevation
aspect
barrier
density
flowline header
e/w distances
quantization tolerances
Reading input files: elevation...
aspect
barrier
Precomputing: tangents
e/w distances
quantization tolerances
re-oriented aspects
Working...
Done%
Writing density file...
Done
Closing files...
Building topology ...
0 primitives registered
0 areas built
0 isles built
Attaching islands:
Attaching centroids: Topology was built.
Number of nodes : 0
Number of primitives: 0
Number of points : 0
Number of lines : 0
Number of boundaries: 0
Number of centroids : 0
Number of areas : 0
Number of isles : 0
Done
De-allocating memory...
Done

What to do? Should I remove all informations about reading, precomputing
and allocating for each raster map? I suggest, that the most verbose
output could look like this:

Opening output files...
WARNING: The vector 'tmp1' already exists and will be overwritten.
Allocating memory for maps
Reading input files
Precomputing
Working
Writing density file
Closing files
Building topology
0 primitives registered
0 areas built
0 isles built
Attaching islands:
Attaching centroids: Topology was built.
Number of nodes : 0
Number of primitives: 0
Number of points : 0
Number of lines : 0
Number of boundaries: 0
Number of centroids : 0
Number of areas : 0
Number of isles : 0
Done
De-allocating memory

what would you say?

jachym
--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://les-ejk.cz/gnupg_public_key/jachym_cepicky-gpg_public_key.asc
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514

6

Hi,

what about this patch?

Some messages changed to G_debug.

Example output:

Reading input files: elevation
Reading input files: aspect
Reading input files: barrier
Precomputing: tangents
Precomputing: e/w distances
Precomputing: quantization tolerances
Precomputing: re-oriented aspects
Working
Writing density file
Closing files

Jachym

On Thu, Oct 19, 2006 at 09:21:04PM +0200, Jachym Cepicky wrote:

hallo,

i do not know, how to solve fprints in r.flow.

The module use it's own function to print messages (diag)

When i removed this function and rewrite everything with help of
G_message, the result is not well-arranged:

r.flow elevin=elevation.dem aspin=aspect barin=roads flout=tmp1
lgout=tmp2 dsout=tmp3 --o
Opening output files...
WARNING: The vector 'tmp1' already exists and will be overwritten.
Done
Allocating memory: elevation
aspect
barrier
density
flowline header
e/w distances
quantization tolerances
Reading input files: elevation...
aspect
barrier
Precomputing: tangents
e/w distances
quantization tolerances
re-oriented aspects
Working...
Done%
Writing density file...
Done
Closing files...
Building topology ...
0 primitives registered
0 areas built
0 isles built
Attaching islands:
Attaching centroids: Topology was built.
Number of nodes : 0
Number of primitives: 0
Number of points : 0
Number of lines : 0
Number of boundaries: 0
Number of centroids : 0
Number of areas : 0
Number of isles : 0
Done
De-allocating memory...
Done

What to do? Should I remove all informations about reading, precomputing
and allocating for each raster map? I suggest, that the most verbose
output could look like this:

Opening output files...
WARNING: The vector 'tmp1' already exists and will be overwritten.
Allocating memory for maps
Reading input files
Precomputing
Working
Writing density file
Closing files
Building topology
0 primitives registered
0 areas built
0 isles built
Attaching islands:
Attaching centroids: Topology was built.
Number of nodes : 0
Number of primitives: 0
Number of points : 0
Number of lines : 0
Number of boundaries: 0
Number of centroids : 0
Number of areas : 0
Number of isles : 0
Done
De-allocating memory

what would you say?

jachym
--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://les-ejk.cz/gnupg_public_key/jachym_cepicky-gpg_public_key.asc
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514

_______________________________________________
grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://les-ejk.cz/gnupg_public_key/jachym_cepicky-gpg_public_key.asc
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514

(attachments)

r.flow.patch (11.5 KB)

7

On Oct 21, 2006, at 8:15 AM, Jachym Cepicky wrote:

Hi,

what about this patch?

Some messages changed to G_debug.

Example output:

Reading input files: elevation
Reading input files: aspect
Reading input files: barrier
Precomputing: tangents
Precomputing: e/w distances
Precomputing: quantization tolerances
Precomputing: re-oriented aspects
Working
Writing density file
Closing files

it looks good to me, Helena

Jachym

On Thu, Oct 19, 2006 at 09:21:04PM +0200, Jachym Cepicky wrote:

hallo,

i do not know, how to solve fprints in r.flow.

The module use it's own function to print messages (diag)

When i removed this function and rewrite everything with help of
G_message, the result is not well-arranged:

r.flow elevin=elevation.dem aspin=aspect barin=roads flout=tmp1
lgout=tmp2 dsout=tmp3 --o
Opening output files...
WARNING: The vector 'tmp1' already exists and will be overwritten.
Done
Allocating memory: elevation
aspect
barrier
density
flowline header
e/w distances
quantization tolerances
Reading input files: elevation...
aspect
barrier
Precomputing: tangents
e/w distances
quantization tolerances
re-oriented aspects
Working...
Done%
Writing density file...
Done
Closing files...
Building topology ...
0 primitives registered
0 areas built
0 isles built
Attaching islands:
Attaching centroids: Topology was built.
Number of nodes : 0
Number of primitives: 0
Number of points : 0
Number of lines : 0
Number of boundaries: 0
Number of centroids : 0
Number of areas : 0
Number of isles : 0
Done
De-allocating memory...
Done

What to do? Should I remove all informations about reading, precomputing
and allocating for each raster map? I suggest, that the most verbose
output could look like this:

Opening output files...
WARNING: The vector 'tmp1' already exists and will be overwritten.
Allocating memory for maps
Reading input files
Precomputing
Working
Writing density file
Closing files
Building topology
0 primitives registered
0 areas built
0 isles built
Attaching islands:
Attaching centroids: Topology was built.
Number of nodes : 0
Number of primitives: 0
Number of points : 0
Number of lines : 0
Number of boundaries: 0
Number of centroids : 0
Number of areas : 0
Number of isles : 0
Done
De-allocating memory

what would you say?

jachym
--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://les-ejk.cz/gnupg_public_key/jachym_cepicky-gpg_public_key.asc
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514

_______________________________________________
grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://les-ejk.cz/gnupg_public_key/jachym_cepicky-gpg_public_key.asc
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514
<r.flow.patch>
_______________________________________________
grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

8

Can we assume that these messages would be output with the --v flag and
suppressed with the --q flag? If so, this looks fine and is a big
improvement.

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution & Social Change
Center for Social Dynamics & Complexity
Arizona State University

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: Jachym Cepicky <jachym.cepicky@centrum.cz>
Date: Sat, 21 Oct 2006 14:15:44 +0200
To: Jachym Cepicky <jachym.cepicky@centrum.cz>
Cc: <grass-dev@grass.itc.it>
Subject: Re: [GRASS-dev] r.flow and G_message

Hi,

what about this patch?

Some messages changed to G_debug.

Example output:

Reading input files: elevation
Reading input files: aspect
Reading input files: barrier
Precomputing: tangents
Precomputing: e/w distances
Precomputing: quantization tolerances
Precomputing: re-oriented aspects
Working
Writing density file
Closing files

Jachym

On Thu, Oct 19, 2006 at 09:21:04PM +0200, Jachym Cepicky wrote:

hallo,

i do not know, how to solve fprints in r.flow.

The module use it's own function to print messages (diag)

When i removed this function and rewrite everything with help of
G_message, the result is not well-arranged:

r.flow elevin=elevation.dem aspin=aspect barin=roads flout=tmp1
lgout=tmp2 dsout=tmp3 --o
Opening output files...
WARNING: The vector 'tmp1' already exists and will be overwritten.
Done
Allocating memory: elevation
aspect
barrier
density
flowline header
e/w distances
quantization tolerances
Reading input files: elevation...
aspect
barrier
Precomputing: tangents
e/w distances
quantization tolerances
re-oriented aspects
Working...
Done%
Writing density file...
Done
Closing files...
Building topology ...
0 primitives registered
0 areas built
0 isles built
Attaching islands:
Attaching centroids: Topology was built.
Number of nodes : 0
Number of primitives: 0
Number of points : 0
Number of lines : 0
Number of boundaries: 0
Number of centroids : 0
Number of areas : 0
Number of isles : 0
Done
De-allocating memory...
Done

What to do? Should I remove all informations about reading, precomputing
and allocating for each raster map? I suggest, that the most verbose
output could look like this:

Opening output files...
WARNING: The vector 'tmp1' already exists and will be overwritten.
Allocating memory for maps
Reading input files
Precomputing
Working
Writing density file
Closing files
Building topology
0 primitives registered
0 areas built
0 isles built
Attaching islands:
Attaching centroids: Topology was built.
Number of nodes : 0
Number of primitives: 0
Number of points : 0
Number of lines : 0
Number of boundaries: 0
Number of centroids : 0
Number of areas : 0
Number of isles : 0
Done
De-allocating memory

what would you say?

jachym
--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://les-ejk.cz/gnupg_public_key/jachym_cepicky-gpg_public_key.asc
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514

_______________________________________________
grass-dev mailing list
grass-dev@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass-dev

--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://les-ejk.cz/gnupg_public_key/jachym_cepicky-gpg_public_key.asc
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514

9

hi,

so the output looks now like this (with --verbose):

Reading input files: elevation
Reading input files: aspect
Reading input files: barrier
Precomputing: tangents
Precomputing: e/w distances
Precomputing: quantization tolerances
Precomputing: re-oriented aspects
Calculating maps ...
10%
Writing density file
Building topology ...

the patch is commited

thank you all for comments

jachym

On Sat, Oct 21, 2006 at 03:25:55PM -0700, Brad Douglas wrote:

On Sat, 2006-10-21 at 14:15 +0200, Jachym Cepicky wrote:
> Hi,
>
> what about this patch?
>
> Some messages changed to G_debug.
>
> Example output:
>
> Reading input files: elevation
> Reading input files: aspect
> Reading input files: barrier
> Precomputing: tangents
> Precomputing: e/w distances
> Precomputing: quantization tolerances
> Precomputing: re-oriented aspects
> Working

I would get rid of the above line unless there is a progress indicator
to follow. Maybe add some ellipses after as an indicator that something
is happening or will happen (ie. 'Working...').

> Writing density file
> Closing files

I'd get rid of the above line, too. No user cares about closing
files. :slight_smile:

Great job! Much nicer.

--
Brad Douglas <rez touchofmadness com> KB8UYR/6
Address: 37.493,-121.924 / WGS84 National Map Corps #TNMC-3785

--
Jachym Cepicky
e-mail: jachym.cepicky@centrum.cz
URL: http://les-ejk.cz
GPG: http://les-ejk.cz/gnupg_public_key/jachym_cepicky-gpg_public_key.asc
-----------------------------------------
OFFICE:
Department of Geoinformation Technologies
Zemedelska 3
613 00, Brno
Czech Republick
e-mail: xcepicky@node.mendelu.cz
URL: http://mapserver.mendelu.cz
Tel.: +420 545 134 514

10

Jachym Cepicky wrote:

so the output looks now like this (with --verbose):

Reading input files: elevation
Reading input files: aspect
Reading input files: barrier
Precomputing: tangents
Precomputing: e/w distances
Precomputing: quantization tolerances
Precomputing: re-oriented aspects
Calculating maps ...
10%
Writing density file
Building topology ...

is "10%" a typo for "100%" or real?

if using G_percent() you need to call it again after the loop has
finished otherwise it stops at 98% (or whatever) and doesn't print the
newline to stderr. IIRC r.terraflow is pretty bad for this mistake.

Hamish

11

Glynn Clements wrote:

Also, I would suggest an additional verbosity level between "message"
and "debug" which would be used for messages which are relatively
unimportant, yet still meant for users rather than developers, with
"debug" being reserved for information which is only useful for, well,
debugging.

along these lines, here's a trick:

G_debug(0, "");

is always printed.

Probably reserve this for very rare cases just before G_fatal_error();
[which exits], if extra messages are needed that can't/shouldn't fit in
the error message.

Hamish