[GRASS5] Re: Automatic gettextizations ; tables and g.html2man

GRASS GIS GRASS GIS developer
1

Hi Daniel,
(cc grass5)

On Fri, Mar 04, 2005 at 01:55:38AM -0500, Daniel Calvelo Aros wrote:

Hi, Markus.

I spent some time estimating the amount of work needed to _-mark all
translatable strings. There is one case that can be fully automated: the
G_(message|warning|fatal_error) functions. There is a mostly automatic case:
description strings for options. There is a half automatic case: fprintf's.

In the attached tarball there are two files. The important one is
gettextize.cmds. It's a howto-commented script that tries to perform the
automatic possible _-markings.

1. For G_*, it's straightforward: find all instances of
G_(message|warning|fatal_error) calls, and enclose the constant strings
therein with _(). I used a find-grep-perl filter to generate a context diff
that is then patch'ed into the source tree. Only non-constant strings are left
to be hand-marked, things like 'G_message(msg)'.

2. For option descriptions, the same principle applies, only care must be
taken with multi-line strings, which I haven't found in the G_* calls.

3. For fprintf calls, it's trickier since they are mostly present in older
code, which doesn't quite follow Radim's proposal of using stderr for anything
informative and stdout for output proper. Also, there are several occurrences
of fprintf being used for debug and exception signaling. In those cases, the
output is meant to be read by the command's developper rather than the user.
what I did was a find-grep filter, followed by hand editing of the resulting
occurrences, followed by patch generation. The hand-edited intermediary file
is 'fprintf_nongettexted', and corresponds to 6.0.0beta2. I'm not sure it's as
useful as the other two conversions.

thanks for your work. I have attached for the list the
gettextize_auto.tgz script for further discussion (which hopefully happens).

Finally, as I was happily perling, I remember having come across a message by
yourself to grass5 list where you sought to have <table>s managed in
g.html2man; I attach a patch that allows for *well-formed* <table> entries to
be translated to tbl markup. (By well-formed I mean using <th></th> as actual
headers, not boldface markers and closing <td> and <tr> correctly.)

Very nice, uploaded to CVS (please verify:
cvs up tools
cd man
make
)

I ran the automatic converter on beta2 sources, and I got some 1700 undefined
entries in grassmods. Scary. I haven't run through it, but it is possible that
several messages are slight variations of each other and should be merged
accordingly. That should go along the *huge* work of going module by module
checking proper usage of output.

In fact, we have to somewhat cleanup the messages to avoid that
it becomes an endless story. This is time consuming, but consider
the time needed to replicate translations for n languages...

I hope this is helpful.

Definitely! Please consider to subscribe to grass5 to participate
in the discussion (changing to digest mode/mime you can keep the number
of mails smaller if needed).

Thanks for your input

Markus

(attachments)

gettextize_auto.tgz (47.8 KB)

2

> I ran the automatic converter on beta2 sources, and I got some 1700
> undefined entries in grassmods. Scary. I haven't run through it, but
> it is possible that several messages are slight variations of each
> other and should be merged accordingly. That should go along the
> *huge* work of going module by module checking proper usage of
> output.

In fact, we have to somewhat cleanup the messages to avoid that
it becomes an endless story. This is time consuming, but consider
the time needed to replicate translations for n languages...

There is probably 500 different versions of "Failed to load map <$MAP>"
which could all be the same..

To complicate matters, for easier debugging I have sometimes made
several identical error messages within a module slightly different.
That way when it fails one can quickly search to where in the code it
failed without all dumping all sorts of debug info out all the time.
e.g. <map> vs [map] or 'single' vs "double" quotes in error.
(I think I did this in r.*.mat for example)

(e.g. r.in.mat, "Please contact the GRASS development team")

I suppose the translation effort is more important than subtle
debugging tricks. Mentioned so you are aware this sort of thing exists.

Hamish

3

[Hamish]

There is probably 500 different versions of "Failed to load map <$MAP>"
which could all be the same..

To complicate matters, for easier debugging I have sometimes made
several identical error messages within a module slightly different.
That way when it fails one can quickly search to where in the code it
failed without all dumping all sorts of debug info out all the time.
e.g. <map> vs [map] or 'single' vs "double" quotes in error.
(I think I did this in r.*.mat for example)

Mmmmh... Would it make sense to have a G_debug macro/function (in gis.h? not
sure...) to help in that? I was thinking of something like providing

#define G_debug(msg)\
#ifdef DEBUG\
fprintf(stderr,"[%s:%s] ",__FILE__, __LINE__);\
#ifdef __GNUC__\
fprintf(stderr,"in %s : ", __FUNCTION__);\
#endif\
fprintf(stderr, ##msg );\
#endif

That would enable:

1) to locate precisely those debug messages and not worry about them in
translations

2) to not type all those 'fprintf's :slight_smile:

BTW such a macro could be extended to support verbose modes and so on.

(e.g. r.in.mat, "Please contact the GRASS development team")

I suppose the translation effort is more important than subtle
debugging tricks. Mentioned so you are aware this sort of thing exists.

Thanks for the tip. Another annoyance in translations is several lines of
fprintfs instead of one multiline string. That's quite uncomfortable when
editing po-files. I'll look into it, it must be detectable automatically.

Daniel.

4

Mmmmh... Would it make sense to have a G_debug macro/function (in
gis.h? not sure...) to help in that? I was thinking of something like
providing

Radim has already provided this :wink:

lib/gis/debug.c
/*
*
* G_debug (int level, char *msg)
*
* Print debugging message if environment variable GRASS_DEBUG_LEVEL
* is set to level equal or greater
*
* Levels: (recommended levels)
* 1 - message is printed once or few times per module
* 3 - each row (raster) or line (vector)
* 5 - each cell (raster) or point (vector)
*/

example:

int cat;
G_debug(3, "cat=%d", cat);

GRASS> g.gisenv set="DEBUG=3"
GRASS> g.module
D3/3: cat=123
GRASS>

This is new for 5.7/6.0 though, so while the new vector modules make
good use of it the old raster and display modules inherited from 5.0 and
earlier use lots of "#ifdef DEBUG" variations. Pre-processing should cut
all these out, so for your translation focusing efforts, perhaps run the
scripts on the cleaned version of the source code? (man gcc)

On the positive side, a lot of those error messages are there in case
there is a bug in the program, hopefully they will never be shown. Thus
they are very low priority for selective translating.

Fixing things to be translation friendly in all modules is a huge amount
of work. I think a good way to start is to define some best practices
and add them into the SUBMITTING file. Then at least whenever a module
gets a re-write or a new module appears it will be easy.

Also new for 5.7/6.0 is G_define_standard_option(). Updating modules to
use this instead of a custom version every time would also make your
efforts a bit simpler.

Another annoyance in translations is several lines
of fprintfs instead of one multiline string. That's quite
uncomfortable when editing po-files. I'll look into it, it must be
detectable automatically.

Does this work for all compilers?

search:
grep -A1 printf | sed | uniq --repeated

or something like that?

Hamish

5

G_debug (int level, char *msg) should be changed to macro with suggested line/row additions and the macro should also do nothing (don't call any function) if GRASS is compiled without debug, that should also speed it up.

Radim

Hamish wrote:

Mmmmh... Would it make sense to have a G_debug macro/function (in
gis.h? not sure...) to help in that? I was thinking of something like
providing

Radim has already provided this :wink:

lib/gis/debug.c
/*
*
* G_debug (int level, char *msg)
*
* Print debugging message if environment variable GRASS_DEBUG_LEVEL
* is set to level equal or greater
*
* Levels: (recommended levels)
* 1 - message is printed once or few times per module
* 3 - each row (raster) or line (vector)
* 5 - each cell (raster) or point (vector)
*/

example:

int cat;
G_debug(3, "cat=%d", cat);

GRASS> g.gisenv set="DEBUG=3"
GRASS> g.module
D3/3: cat=123
GRASS>

This is new for 5.7/6.0 though, so while the new vector modules make
good use of it the old raster and display modules inherited from 5.0 and
earlier use lots of "#ifdef DEBUG" variations. Pre-processing should cut
all these out, so for your translation focusing efforts, perhaps run the
scripts on the cleaned version of the source code? (man gcc)

On the positive side, a lot of those error messages are there in case
there is a bug in the program, hopefully they will never be shown. Thus
they are very low priority for selective translating.

Fixing things to be translation friendly in all modules is a huge amount
of work. I think a good way to start is to define some best practices
and add them into the SUBMITTING file. Then at least whenever a module
gets a re-write or a new module appears it will be easy.

Also new for 5.7/6.0 is G_define_standard_option(). Updating modules to
use this instead of a custom version every time would also make your
efforts a bit simpler.

Another annoyance in translations is several lines
of fprintfs instead of one multiline string. That's quite
uncomfortable when editing po-files. I'll look into it, it must be
detectable automatically.

Does this work for all compilers?

search:
grep -A1 printf | sed | uniq --repeated

or something like that?

Hamish

_______________________________________________
grass5 mailing list
grass5@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass5

6

On Mon, Mar 07, 2005 at 05:29:06PM +1300, Hamish wrote:

lib/gis/debug.c
/*
*
* G_debug (int level, char *msg)
*
* Print debugging message if environment variable GRASS_DEBUG_LEVEL
* is set to level equal or greater
*
* Levels: (recommended levels)
* 1 - message is printed once or few times per module
* 3 - each row (raster) or line (vector)
* 5 - each cell (raster) or point (vector)
*/

Just a side-note:
Please use doxygen style for commenting functions. Otherwise
the function won't appear in the automated programmer's manual.

I updated around 800-1000 comments and I am a bit tired to continue.
So please, follow doxygen style, there are many examples in
the libraries already.

Thanks

Markus

7

[Markus]

Just a side-note:
Please use doxygen style for commenting functions. Otherwise
the function won't appear in the automated programmer's manual.

SUBMITTING, point 10 ?

I updated around 800-1000 comments and I am a bit tired to continue.
So please, follow doxygen style, there are many examples in
the libraries already.

Ok.

8

Daniel Calvelo Aros wrote:

Mmmmh... Would it make sense to have a G_debug macro/function (in gis.h? not
sure...) to help in that? I was thinking of something like providing

#define G_debug(msg)\
#ifdef DEBUG\
fprintf(stderr,"[%s:%s] ",__FILE__, __LINE__);\
#ifdef __GNUC__\
fprintf(stderr,"in %s : ", __FUNCTION__);\
#endif\
fprintf(stderr, ##msg );\
#endif

Including preprocessor directives in a macro definition isn't valid
ANSI C; you would need to move the #define into each branch of the
conditional, e.g.:

#ifdef DEBUG
# ifdef __GNUC__
# define G_debug(msg) fprintf(stderr,"[%s:%s] in %s : %s\n",__FILE__, __LINE__, __FUNCTION__, (msg))
# else
# define G_debug(msg) fprintf(stderr,"[%s:%s] : %s\n",__FILE__, __LINE__, (msg))
# endif
#endif

But the general idea of using __FILE__ etc is sound.

--
Glynn Clements <glynn@gclements.plus.com>