[GRASS-dev] [GRASS GIS] #3508: grass start up: buggy to_text_string

GDAL · March 2, 2018, 3:09pm

#3508: grass start up: buggy to_text_string
-------------------------------+-------------------------
Reporter: martinl | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.2.3
Component: Startup | Version: svn-trunk
Keywords: wingrass, unicode | CPU: Unspecified
Platform: Unspecified |
-------------------------------+-------------------------
GRASS fails even start on Windows when username contains non-ascii
characters

{{{
Traceback (most recent call last):
   File "C:\Program Files\GRASS GIS 7.4.0\etc\grass74.py", line 2003, in
<module>
     main()
   File "C:\Program Files\GRASS GIS 7.4.0\etc\grass74.py", line 1854, in
main
     set_paths(grass_config_dir=grass_config_dir)
   File "C:\Program Files\GRASS GIS 7.4.0\etc\grass74.py", line 620, in
set_paths
     os.environ['MANPATH'] = to_text_string(addons_man_path)
   File "C:\Program Files\GRASS GIS 7.4.0\Python27\lib\os.py", line 420, in
__setitem__
     putenv(key, item)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u017e' in
position 10: ordinal not in range(128)
Press any key to continue . . .
}}}

When I change line source:grass/trunk/lib/init/grass.py#L620

{{{
os.environ['MANPATH'] = to_text_string(addons_man_path)
}}}

to

{{{
os.environ['MANPATH'] = addons_man_path
}}}

then GRASS starts and even MANPATH variable is set up correctly (including
non-ascii characters). Function `to_text_string` is apparently buggy.
Tested on Czech (cp1250) Windows 8.1.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3508>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 2, 2018, 3:15pm

#3508: grass start up: buggy to_text_string
--------------------------+-------------------------------
  Reporter: martinl | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.2.3
Component: Startup | Version: svn-trunk
Resolution: | Keywords: wingrass, unicode
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------------
Changes (by martinl):

* priority: normal => blocker

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3508#comment:1>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 2, 2018, 3:18pm

#3508: grass start up: buggy to_text_string
--------------------------+-------------------------------
  Reporter: martinl | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.2.3
Component: Startup | Version: svn-trunk
Resolution: | Keywords: wingrass, unicode
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------------

Comment (by martinl):

`to_text_string` has been introduced 3 years ago in r65780

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3508#comment:2>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 2, 2018, 3:20pm

#3508: grass start up: buggy to_text_string
--------------------------+-------------------------------
  Reporter: martinl | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.2.3
Component: Startup | Version: svn-trunk
Resolution: | Keywords: wingrass, unicode
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------------

Comment (by wenzeslaus):

Right. The code is similar to `_get_encoding()/decode()/encode()` group in
source:grass/trunk/lib/python/script/utils.py#L159 from r65787. Do these
functions work for you (assuming you can get around the startup issue)?
All is basically based around `locale.getdefaultlocale()[1]`.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3508#comment:3>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 2, 2018, 3:35pm

#3508: grass start up: buggy to_text_string
--------------------------+-------------------------------
  Reporter: martinl | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.2.3
Component: Startup | Version: svn-trunk
Resolution: | Keywords: wingrass, unicode
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------------

Comment (by mlennert):

I think this is a classical case of it being better not to mess around
with the original byte string. All we do is taking a string from an
environment variable, adding two new path elements and putting it back
into another environment variable:

{{{
addon_base = os.getenv('GRASS_ADDON_BASE')
[...]
addons_man_path = os.path.join(addon_base, 'docs', 'man')
os.environ['MANPATH'] = to_text_string(addons_man_path)
}}}

Do we really have to worry about encoding/decoding in that context ?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3508#comment:4>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 3, 2018, 9:09am

#3508: grass start up: buggy to_text_string
--------------------------+-------------------------------
  Reporter: martinl | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.2.3
Component: Startup | Version: svn-trunk
Resolution: | Keywords: wingrass, unicode
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------------

Comment (by martinl):

Replying to [comment:4 mlennert]:
> Do we really have to worry about encoding/decoding in that context ?

Encoding/decoding doesn't seems to be needed in this case.

{{{
addons_man_path = os.path.join(addon_base, 'docs', 'man')
}}}

perfectly works on my testing computer.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3508#comment:5>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 3, 2018, 10:10am

#3508: grass start up: buggy to_text_string
--------------------------+-------------------------------
  Reporter: martinl | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.2.3
Component: Startup | Version: svn-trunk
Resolution: | Keywords: wingrass, unicode
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------------

Comment (by hellik):

Replying to [comment:5 martinl]:
> Replying to [comment:4 mlennert]:
> > Do we really have to worry about encoding/decoding in that context ?
>
> Encoding/decoding doesn't seems to be needed in this case.
>
> {{{
> addons_man_path = os.path.join(addon_base, 'docs', 'man')
> }}}
>
> perfectly works on my testing computer.

a related ticket: #3500

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3508#comment:6>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 16, 2018, 12:07pm

#3508: grass start up: buggy to_text_string
--------------------------+-------------------------------
  Reporter: martinl | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.2.3
Component: Startup | Version: svn-trunk
Resolution: | Keywords: wingrass, unicode
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------------

Comment (by martinl):

In [changeset:"72374" 72374]:
{{{
#!CommitTicketReference repository="" revision="72374"
disable to_text_string() in grass.py, see #3508
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3508#comment:7>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 16, 2018, 12:08pm

#3508: grass start up: buggy to_text_string
--------------------------+-------------------------------
  Reporter: martinl | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.2.3
Component: Startup | Version: svn-trunk
Resolution: | Keywords: wingrass, unicode
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------------

Comment (by martinl):

`to_text_string()` disabled in r72374. Planning backport to 7.4 and 7.2
branches, any objections?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3508#comment:8>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 18, 2018, 5:41pm

#3508: grass start up: buggy to_text_string
--------------------------+-------------------------------
  Reporter: martinl | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.2.3
Component: Startup | Version: svn-trunk
Resolution: | Keywords: wingrass, unicode
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------------

Comment (by mlennert):

Replying to [comment:8 martinl]:
> `to_text_string()` disabled in r72374. Planning backport to 7.4 and 7.2
branches, any objections?

None from me.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3508#comment:9>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 20, 2018, 10:52am

#3508: grass start up: buggy to_text_string
--------------------------+-------------------------------
  Reporter: martinl | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.2.3
Component: Startup | Version: svn-trunk
Resolution: | Keywords: wingrass, unicode
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------------

Comment (by martinl):

In [changeset:"72401" 72401]:
{{{
#!CommitTicketReference repository="" revision="72401"
disable to_text_string() in grass.py, see #3508 (merge r72374 from trunk)
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3508#comment:10>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 20, 2018, 11:00am

#3508: grass start up: buggy to_text_string
--------------------------+-------------------------------
  Reporter: martinl | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: blocker | Milestone: 7.2.3
Component: Startup | Version: svn-trunk
Resolution: | Keywords: wingrass, unicode
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------------

Comment (by martinl):

In [changeset:"72402" 72402]:
{{{
#!CommitTicketReference repository="" revision="72402"
disable to_text_string() in grass.py, see #3508 (merge r72374 from trunk)
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3508#comment:11>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 20, 2018, 11:05am

#3508: grass start up: buggy to_text_string
--------------------------+-------------------------------
  Reporter: martinl | Owner: grass-dev@…
      Type: defect | Status: closed
  Priority: blocker | Milestone: 7.2.3
Component: Startup | Version: svn-trunk
Resolution: fixed | Keywords: wingrass, unicode
       CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------------
Changes (by martinl):

* status: new => closed
* resolution: => fixed

Comment:

Fixed in all affected branches: grass74 and grass72.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3508#comment:12>
GRASS GIS <https://grass.osgeo.org>