[GRASS-dev] winGRASS: enabling non-english languages?

Hi,

we have tried the latest winGRASS from today which has nls enabled
according to Helmut. The .mo files are present in the install directory.

The computer (XP) is runnig in Japanese locale but still the English
screen comes up. If I do the same on Linux, I get the Japanese
startup screen.

How to debug this problem? It would be essential IMHO to get nls
working (and I feel that there is a rather trivial obstacle in the way).

Markus

Markus wrote:

we have tried the latest winGRASS from today which has nls
enabled according to Helmut. The .mo files are present in the
install directory.

The computer (XP) is runnig in Japanese locale but still
the English screen comes up. If I do the same on Linux, I get the
Japanese startup screen.

How to debug this problem? It would be essential IMHO to
get nls working (and I feel that there is a rather trivial obstacle
in the way).

https://trac.osgeo.org/grass/changeset/40063

??
H

Markus Neteler wrote:

we have tried the latest winGRASS from today which has nls enabled
according to Helmut. The .mo files are present in the install directory.

The computer (XP) is runnig in Japanese locale but still the English
screen comes up. If I do the same on Linux, I get the Japanese
startup screen.

How to debug this problem? It would be essential IMHO to get nls
working (and I feel that there is a rather trivial obstacle in the way).

Are LANG or LC_MESSAGES set?

If I set LANG=ja, precisely one of the menu entries gets
translated: Help (ヘルプ).

FWIW, I've verified that this is actually getting translated by the
_() in menudata.py (I thought that the "help" menu might be a special
case hard-wired into wxWidgets).

If I use e.g. LANG=fr or LANG=de, I see far more translated entries.
Okay, those files are larger, but grasswxpy_ja.po still has 471
translations. However: grasswxpy_ja.po has 445 occurrences of "fuzzy",
and ja/LC_MESSAGES/grasswxpy.mo is tiny (4608 bytes, versus 40617
bytes for fr).

Are "fuzzy" translations ignored when generating the .mo file?

--
Glynn Clements <glynn@gclements.plus.com>

On Wed, Jan 13, 2010 at 8:28 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

we have tried the latest winGRASS from today which has nls enabled
according to Helmut. The .mo files are present in the install directory.

The computer (XP) is runnig in Japanese locale but still the English

This Windows-XP machine is all-Japanese.

screen comes up. If I do the same on Linux, I get the Japanese
startup screen.

How to debug this problem? It would be essential IMHO to get nls
working (and I feel that there is a rather trivial obstacle in the way).

Are LANG or LC_MESSAGES set?

We verified in the GRASS-CMD line: they are not set.
So that will be the issue.

If I set LANG=ja, precisely one of the menu entries gets
translated: Help (ヘルプ).

Here it fails because GRASS is installed in "Program Files".
g.gui wxpython
bails out with a related error.

Of course I had to set GRASS_PYTHON=python manually before but
due to the space problem it doesn't come up.
At least we seem to be close.

FWIW, I've verified that this is actually getting translated by the
_() in menudata.py (I thought that the "help" menu might be a special
case hard-wired into wxWidgets).

Yes, Quit and Help refuse to be translated. We''ll see later.

If I use e.g. LANG=fr or LANG=de, I see far more translated entries.
Okay, those files are larger, but grasswxpy_ja.po still has 471
translations. However: grasswxpy_ja.po has 445 occurrences of "fuzzy",
and ja/LC_MESSAGES/grasswxpy.mo is tiny (4608 bytes, versus 40617
bytes for fr).

I just checked in a new transfer of translated messages for JA.
They are currently under review at Osaka City University to get
rid of fuzzy and to fix the wrong translations.

Are "fuzzy" translations ignored when generating the .mo file?

Yes.

Markus

On Wed, Jan 13, 2010 at 9:25 AM, Markus Neteler <neteler@osgeo.org> wrote:

On Wed, Jan 13, 2010 at 8:28 AM, Glynn Clements wrote:

Markus Neteler wrote:

we have tried the latest winGRASS from today which has nls enabled

...

If I set LANG=ja, precisely one of the menu entries gets
translated: Help (ヘルプ).

Here it fails because GRASS is installed in "Program Files".
g.gui wxpython
bails out with a related error.

... which was:

GRASS 6.4.0svn (demolocation)> g.gui wxpython
wxgui: can't open file 'C:/Program': [Errno 2] No such file or directory

Any troubles in

        sprintf(progname, "%s/etc/wxpython/wxgui.py", G_gisbase());

?

Of course I had to set GRASS_PYTHON=python manually before but
due to the space problem it doesn't come up.
At least we seem to be close.

Markus

Markus Neteler wrote:

Here it fails because GRASS is installed in "Program Files".
g.gui wxpython
bails out with a related error.

What's the error?

We might need to change G_spawn() to quote arguments. The MSDN
documentation for the _spawn* functions suggests that it simply
concatenates the arguments with spaces in between, and that the caller
must perform quoting.

  http://msdn.microsoft.com/en-us/library/20y988d2(VS.80).aspx

I've dealt with the issue in 7.0 by using G_spawn_ex() instead, and
implementing that using CreateProcess() (so redirection works on
Windows). Eventually, G_spawn_ex() will be renamed to G_spawn().

Even if we continue to use _spawn*, the quoting code can be taken from
7.0; see escaped() in lib/gis/spawn.c. The code is actually quite
tricky, due to the bizarre quoting rules:

  http://msdn.microsoft.com/en-us/library/17w5ykft.aspx

Of course I had to set GRASS_PYTHON=python manually before but
due to the space problem it doesn't come up.
At least we seem to be close.

If the problem is with GISBASE, you could try manually changing it to
use "progra~1" in place of "Program Files" (more generally, "dir /x"
lists the short names if you need them for other cases).

> Are "fuzzy" translations ignored when generating the .mo file?

Yes.

Ah; that would explain why there are hardly any Japanese
translations; nearly all of them are "fuzzy".

--
Glynn Clements <glynn@gclements.plus.com>

On Wed, Jan 13, 2010 at 8:25 PM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

Here it fails because GRASS is installed in "Program Files".
g.gui wxpython
bails out with a related error.

What's the error?

It is:

GRASS 6.4.0svn (demolocation)> g.gui wxpython
wxgui: can't open file 'C:/Program': [Errno 2] No such file or directory

We might need to change G_spawn() to quote arguments. The MSDN
documentation for the _spawn* functions suggests that it simply
concatenates the arguments with spaces in between, and that the caller
must perform quoting.

   http://msdn.microsoft.com/en-us/library/20y988d2%28VS.80%29.aspx

I've dealt with the issue in 7.0 by using G_spawn_ex() instead, and
implementing that using CreateProcess() (so redirection works on
Windows). Eventually, G_spawn_ex() will be renamed to G_spawn().

Even if we continue to use _spawn*, the quoting code can be taken from
7.0; see escaped() in lib/gis/spawn.c. The code is actually quite
tricky, due to the bizarre quoting rules:

   http://msdn.microsoft.com/en-us/library/17w5ykft.aspx

Would it be possible for you to suggest a patch?

Of course I had to set GRASS_PYTHON=python manually before but
due to the space problem it doesn't come up.
At least we seem to be close.

If the problem is with GISBASE, you could try manually changing it to
use "progra~1" in place of "Program Files" (more generally, "dir /x"
lists the short names if you need them for other cases).

OK, we'll try, thanks.

> Are "fuzzy" translations ignored when generating the .mo file?

Yes.

Ah; that would explain why there are hardly any Japanese
translations; nearly all of them are "fuzzy".

Yes, being worked on right now.

Markus

Markus Neteler wrote:

> Even if we continue to use _spawn*, the quoting code can be taken from
> 7.0; see escaped() in lib/gis/spawn.c. The code is actually quite
> tricky, due to the bizarre quoting rules:
>
> http://msdn.microsoft.com/en-us/library/17w5ykft.aspx

Would it be possible for you to suggest a patch?

I'll look into it.

> If the problem is with GISBASE, you could try manually changing it to
> use "progra~1" in place of "Program Files" (more generally, "dir /x"
> lists the short names if you need them for other cases).

OK, we'll try, thanks.

FWIW, I'm not suggesting this as a "solution", but as a workaround for
that particular issue so that it doesn't hold up other testing.

--
Glynn Clements <glynn@gclements.plus.com>

Glynn Clements wrote:

> > Even if we continue to use _spawn*, the quoting code can be taken from
> > 7.0; see escaped() in lib/gis/spawn.c. The code is actually quite
> > tricky, due to the bizarre quoting rules:
> >
> > http://msdn.microsoft.com/en-us/library/17w5ykft.aspx
>
> Would it be possible for you to suggest a patch?

I'll look into it.

Copying spawn.c from 7.0 seems to work fine. But it's a significant
change, so it should be tested as thoroughly as possible. Also, fixing
G_spawn() doesn't help with modules which use system() or popen(); I
quite recently replaced most of those in 7.0.

BTW: in looking for test cases, I noticed that neither "g.list -f" nor
"g.mlist -f" work on Windows. E.g.:

    sprintf(lister, "%s/etc/lister/%s", G_gisbase(),
      list[n].element[0]);

    if (access(lister, 1) == 0) { /* execute permission? */
  G_spawn(lister, lister, opt.mapset->answer, NULL);
  continue;
    }

Because the path lacks the .exe suffix, the access() call fails.
G_spawn() itself doesn't need the .exe suffix, but it never gets that
far.

--
Glynn Clements <glynn@gclements.plus.com>

On Thu, Jan 14, 2010 at 1:22 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Glynn Clements wrote:

> > Even if we continue to use _spawn*, the quoting code can be taken from
> > 7.0; see escaped() in lib/gis/spawn.c. The code is actually quite
> > tricky, due to the bizarre quoting rules:
> >
> > http://msdn.microsoft.com/en-us/library/17w5ykft.aspx
>
> Would it be possible for you to suggest a patch?

I'll look into it.

Copying spawn.c from 7.0 seems to work fine. But it's a significant
change, so it should be tested as thoroughly as possible. Also, fixing
G_spawn() doesn't help with modules which use system() or popen(); I
quite recently replaced most of those in 7.0.

So far I am not able to compile winGRASS myself, but I am ready to test
if someone can compile it with spawn.c copied from G7 to 6.4.

BTW: in looking for test cases, I noticed that neither "g.list -f" nor
"g.mlist -f" work on Windows. E.g.:

sprintf(lister, "%s/etc/lister/%s", G_gisbase(),
list[n].element[0]);

if (access(lister, 1) == 0) { /* execute permission? */
G_spawn(lister, lister, opt.mapset->answer, NULL);
continue;
}

Because the path lacks the .exe suffix, the access() call fails.
G_spawn() itself doesn't need the .exe suffix, but it never gets that
far.

Would this be hell?

#ifndef __MINGW32__
    sprintf(lister, "%s/etc/lister/%s", G_gisbase(),
#else
    sprintf(lister, "%s/etc/lister.exe/%s", G_gisbase(),
#endif
           list[n].element[0]);

Typical MN approach :slight_smile:

?
Markus

On Wed, Jan 13, 2010 at 8:28 AM, Glynn Clements wrote:

Markus Neteler wrote:

...

The computer (XP) is runnig in Japanese locale but still the English
screen comes up.

...

Are LANG or LC_MESSAGES set?

When defining these two variables manually on cmd line, the
Japanese translation comes up (working around the other
g.gui problems): the word "Help" is already translated.

It does not come up when starting the GUI from the
Windows menu (so, also here the variables are undefined).

Summary: these two variables are never defined in the
.bat files. Where should it go? Especially: how? I guess
that the current locale info needs to be fetched somehow
to define LANG/LC_MESSAGES properly.

Markus

Markus Neteler wrote:

> Because the path lacks the .exe suffix, the access() call fails.
> G_spawn() itself doesn't need the .exe suffix, but it never gets that
> far.

Would this be hell?

#ifndef __MINGW32__
    sprintf(lister, "%s/etc/lister/%s", G_gisbase(),
#else
    sprintf(lister, "%s/etc/lister.exe/%s", G_gisbase(),
#endif

The .exe goes on the end:

#ifndef __MINGW32__
    sprintf(lister, "%s/etc/lister/%s", G_gisbase(),
#else
    sprintf(lister, "%s/etc/lister/%s.exe", G_gisbase(),
#endif

Except ... this requires that the lister is an executable; there's no
fundamental reason why it couldn't be a script, other than the
access() check (well: that, and the fact that G_spawn() etc don't use
the shell currently).

--
Glynn Clements <glynn@gclements.plus.com>

On Thu, Jan 14, 2010 at 7:41 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

> Because the path lacks the .exe suffix, the access() call fails.
> G_spawn() itself doesn't need the .exe suffix, but it never gets that
> far.

Would this be hell?

#ifndef __MINGW32__
sprintf(lister, "%s/etc/lister/%s", G_gisbase(),
#else
sprintf(lister, "%s/etc/lister.exe/%s", G_gisbase(),
#endif

The .exe goes on the end:

Err, sure.

#ifndef __MINGW32__
sprintf(lister, "%s/etc/lister/%s", G_gisbase(),
#else
sprintf(lister, "%s/etc/lister/%s.exe", G_gisbase(),
#endif

Except ... this requires that the lister is an executable; there's no
fundamental reason why it couldn't be a script, other than the
access() check (well: that, and the fact that G_spawn() etc don't use
the shell currently).

So the question is if fixing the problem for g.[m]list -f brings us a big
deal of new troubles. The question is how often -f is used...

Despite that, someone will need to test the copying spawn.c from 7.0
before doing that in 6.4-SVN.

Markus

Markus Neteler wrote:

> Are LANG or LC_MESSAGES set?

When defining these two variables manually on cmd line, the
Japanese translation comes up (working around the other
g.gui problems): the word "Help" is already translated.

It does not come up when starting the GUI from the
Windows menu (so, also here the variables are undefined).

Summary: these two variables are never defined in the
.bat files. Where should it go? Especially: how? I guess
that the current locale info needs to be fetched somehow
to define LANG/LC_MESSAGES properly.

Windows doesn't set these variables, and doesn't use the ISO 639
language codes or the ISO 3166 country codes.

You can retrieve the current user's locale from the registry key:

  HKEY_CURRENT_USER\Control Panel\International\Locale

This will be a string (REG_SZ) value consisting of a hexadecimal
locale code. Or it can be retrieved (as an integer) via
GetUserDefaultLCID():

  http://msdn.microsoft.com/en-us/library/dd318135(VS.85).aspx

A listing of the codes can be found at:

  http://msdn.microsoft.com/en-us/library/dd318693(VS.85).aspx

Better still, a mapping from the Microsoft codes to Unix-style locale
names can be found in the windows_locale variable in Python's
locale.py.

In Python, you can call locale.getdefaultlocale(), which will return a
tuple of the LANG code and the Windows codepage, e.g.:

  >>> import locale
  >>> print locale.getdefaultlocale()
  ('en_GB', 'cp1252')

OTOH, trying to do this from within Init.bat is probably less than
ideal. The attached program (link with -lkernel32) queries the locale,
converts it to the Unix format and writes it to stdout.

--
Glynn Clements <glynn@gclements.plus.com>

(attachments)

winlocale.c (8.34 KB)

On Thu, Jan 14, 2010 at 8:51 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

...

Summary: these two variables are never defined in the
.bat files. Where should it go? Especially: how? I guess
that the current locale info needs to be fetched somehow
to define LANG/LC_MESSAGES properly.

Windows doesn't set these variables, and doesn't use the ISO 639
language codes or the ISO 3166 country codes.

You can retrieve the current user's locale from the registry key:

   HKEY\_CURRENT\_USER\\Control Panel\\International\\Locale

This will be a string (REG_SZ) value consisting of a hexadecimal
locale code. Or it can be retrieved (as an integer) via
GetUserDefaultLCID():

   http://msdn.microsoft.com/en-us/library/dd318135%28VS.85%29.aspx

A listing of the codes can be found at:

   http://msdn.microsoft.com/en-us/library/dd318693%28VS.85%29.aspx

Better still, a mapping from the Microsoft codes to Unix-style locale
names can be found in the windows_locale variable in Python's
locale.py.

In Python, you can call locale.getdefaultlocale(), which will return a
tuple of the LANG code and the Windows codepage, e.g.:

   &gt;&gt;&gt; import locale
   &gt;&gt;&gt; print locale\.getdefaultlocale\(\)
   \(&#39;en\_GB&#39;, &#39;cp1252&#39;\)

Perhaps instead of getdefaultlocale() a user selectable locale should be
implemented (see other email thread), say, from a list of available locales,
or at least non-english (whatever it is on the user's computer) and EN
locale.

OTOH, trying to do this from within Init.bat is probably less than
ideal. The attached program (link with -lkernel32) queries the locale,
converts it to the Unix format and writes it to stdout.

This works (of course). The point is that translations should be enabled
also for CMD line usage which requires as far as I understand a change
in Init.bat (?).

Markus

Markus wrote:

Perhaps instead of getdefaultlocale() a user selectable locale should be
implemented (see other email thread), say, from a list of available locales,
or at least non-english (whatever it is on the user's computer) and EN
locale.

how about a drop-down menu on the GUI startup window sort of like what
Lorenzo has on his OSX build? The value could be written to the .grassrc6
file to be sticky & be used by terminal-only sessions (enviro vars set by
Init.sh if .grassrc6 lang hint is present). If unset the start-up screen
defaults to whatever enviro vars are set to, or if unset then English?

Hamish

Markus Neteler wrote:

Perhaps instead of getdefaultlocale() a user selectable locale should be
implemented (see other email thread), say, from a list of available locales,
or at least non-english (whatever it is on the user's computer) and EN
locale.

I don't know of any other application which does this. Normally, the
application just uses the user's locale (or en_US if the user's locale
isn't supported).

> OTOH, trying to do this from within Init.bat is probably less than
> ideal. The attached program (link with -lkernel32) queries the locale,
> converts it to the Unix format and writes it to stdout.

This works (of course). The point is that translations should be enabled
also for CMD line usage which requires as far as I understand a change
in Init.bat (?).

On Windows, Init.bat should set LANG (or at least LC_MESSAGES and
LC_CTYPE). But LANG needs to be in Unix format (e.g. "ja_JP"), while
Windows provides a numeric code. So we need some way to get the
appropriate LANG value from the Windows locale setting.

--
Glynn Clements <glynn@gclements.plus.com>

On Windows, Init.bat should set LANG (or at least LC_MESSAGES and
LC_CTYPE). But LANG needs to be in Unix format (e.g. "ja_JP"), while
Windows provides a numeric code. So we need some way to get the
appropriate LANG value from the Windows locale setting.

FYI: http://msdn.microsoft.com/en-us/goglobal/bb964664.aspx

On Sat, Jan 16, 2010 at 6:14 AM, Hamish <hamish_b@yahoo.com> wrote:

Markus wrote:

Perhaps instead of getdefaultlocale() a user selectable locale should be
implemented (see other email thread), say, from a list of available locales,
or at least non-english (whatever it is on the user's computer) and EN
locale.

[Martin has kindly added
  self.locale = wx.Locale(language = wx.LANGUAGE_DEFAULT)
which works and which reacts on $LANG etc environmental
variables. I was wrong in my assumtion that it would be a static
definition.]

how about a drop-down menu on the GUI startup window sort of like what
Lorenzo has on his OSX build? The value could be written to the .grassrc6
file to be sticky & be used by terminal-only sessions (enviro vars set by
Init.sh if .grassrc6 lang hint is present). If unset the start-up screen
defaults to whatever enviro vars are set to, or if unset then English?

Sounds very good to me, but as you suggested elsewhere perhaps
better in
Config->GRASS working environment-> Toggle English/default language

as a GRASS session variable?

Markus

On Sat, Jan 16, 2010 at 6:42 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus:

The point is that translations should be enabled
also for CMD line usage which requires as far as I understand a change
in Init.bat (?).

On Windows, Init.bat should set LANG (or at least LC_MESSAGES and
LC_CTYPE). But LANG needs to be in Unix format (e.g. "ja_JP"), while
Windows provides a numeric code. So we need some way to get the
appropriate LANG value from the Windows locale setting.

Thanks for r40533 which I have backported to 6.4 (r40540) and 6.5 (r40539).

Markus