#2727: wxGUI startup: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal
not in range(128)
-------------------------+-------------------------
Reporter: mlennert | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.0.2
Component: wxGUI | Version: svn-trunk
Keywords: encoding | CPU: Unspecified
Platform: Unspecified |
-------------------------+-------------------------
With a just freshly checked out trunk, and LANGUAGE=fr_BE, I get the
following error when trying to launch the wxgui:
{{{
Impossible d'analyser le fichier XML des outils utilisateur. Chargement de
l'arbre des modules par défaut.
Raison: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal not
in range(128)
Traceback (most recent call last):
File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-unknown-
linux-gnu/gui/wxpython/lmgr/menudata.py", line 62, in __init__
MenuTreeModelBuilder.__init__(self, filename,
expandAddons=expandAddons)
File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-unknown-
linux-gnu/gui/wxpython/core/menutree.py", line 66, in __init__
expAddons(xmlTree)
File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-unknown-
linux-gnu/gui/wxpython/core/toolboxes.py", line 346, in expandAddons
_expandRuntimeModules(root, loadMetadata=True)
File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-unknown-
linux-gnu/gui/wxpython/core/toolboxes.py", line 596, in
_expandRuntimeModules
desc, keywords = _loadMetadata(name)
File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-unknown-
linux-gnu/gui/wxpython/core/toolboxes.py", line 631, in _loadMetadata
e = _encode_string(e.value)
File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-unknown-
linux-gnu/gui/wxpython/core/toolboxes.py", line 114, in _encode_string
return EncodeString(string)
File "/data/home/mlennert/SRC/GRASS/grass_trunk/dist.x86_64-unknown-
linux-gnu/gui/wxpython/core/gcmd.py", line 92, in EncodeString
return string.encode(_enc)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 15:
ordinal not in range(128)
}}}
We just cannot seem to get rid of these kinds of errors. I think this
should be a top priority for work on the wxGUI in the near future as this
is a severe usability issue for many people around the world.
#2727: wxGUI startup: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal
not in range(128)
--------------------------+-------------------------
Reporter: mlennert | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.0.2
Component: wxGUI | Version: svn-trunk
Resolution: | Keywords: encoding
CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------
Comment (by wenzeslaus):
Replying to [ticket:2727 mlennert]:
> With a just freshly checked out trunk
Do I understand correctly that this is not happening in 7.0 release
branch?
> We just cannot seem to get rid of these kinds of errors. I think this
should be a top priority for work on the wxGUI in the near future as this
is a severe usability issue for many people around the world.
It would be good if somebody who is actually running in this kind if
environment would do these fixes.
#2727: wxGUI startup: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal
not in range(128)
--------------------------+-------------------------
Reporter: mlennert | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.0.2
Component: wxGUI | Version: svn-trunk
Resolution: | Keywords: encoding
CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------
Comment (by mlennert):
Replying to [comment:1 wenzeslaus]:
> Replying to [ticket:2727 mlennert]:
> > With a just freshly checked out trunk
>
> Do I understand correctly that this is not happening in 7.0 release
branch?
Well, I just checked out release70 and there I get the same error (with
the added info that this seems to be linked to the v.nnstat addon, but in
the command line and it does not prevent the gui from opening:
{{{
v.nnstat: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal not
in range(128)
}}}
So maybe something in trunk that tries to handle addons differently ?
>
> > We just cannot seem to get rid of these kinds of errors. I think this
should be a top priority for work on the wxGUI in the near future as this
is a severe usability issue for many people around the world.
>
> It would be good if somebody who is actually running in this kind if
environment would do these fixes.
I'm willing to help within the limits of available time, but past
discussions have left me in a state of perplexity concering these
problems. AFAIU (and that's quite limited), there seem to be some
fundamental problems with the handling of encoding issues in the GUI.
Maris has tried to find some solutions and Glynn has weighed in with
insights, but I don't think I quite grasp the whole thing.
It seems to me that we need to fundamentally rethink encoding handling,
but that up to now encoding issues have been 'fixed' one by one without
getting at the underlying weaknesses. But maybe I'm completely wrong in
that understanding...
#2727: wxGUI startup: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal
not in range(128)
--------------------------+-------------------------
Reporter: mlennert | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.0.2
Component: wxGUI | Version: svn-trunk
Resolution: | Keywords: encoding
CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------
Comment (by mlennert):
Replying to [comment:3 annakrat]:
> Please try r66036.
Works, thanks !
> Currently (quite recent change r64834) translatable strings return byte
strings and we have to respect it in the code.
Yes. This change (and the ticket it relates to) clearly show one of the
problems we currently have in grass development: with the growing
importance of the wxGUI and of Python scripting, I have the feeling that
choices at one point in the code have more and more impact on other parts
of the code. Development at module level does not have this kind of impact
and in my (perhaps erroneous) perception C-library changes were done by
fewer people, possibly with better understanding of the consequences.
With Python opening development up to more people, and with the growing
importance of the GUI, we might need a more centralised ruleset of how to
do certain important things in order to ensure that we keep the code
coherent.
>
> It's difficult to define any rules when things change all the time...
That's why I've been pleading for a more fundamental reflection about how
to approach encoding issues in the wxGUI in order to define one way to
things once and for all without trying to solve this case-by-case. The
trouble seems to be that most of us do not really fundamentally understand
how string handling works and what different choices involve.
But then again, maybe this is already solved... More importantly,I don't
have the time at this stage to delve into the matter enough to be the
motor of this, so I will shut up.
#2727: wxGUI startup: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal
not in range(128)
--------------------------+-------------------------
Reporter: mlennert | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.0.2
Component: wxGUI | Version: svn-trunk
Resolution: | Keywords: encoding
CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------
Comment (by annakrat):
Replying to [comment:6 annakrat]:
> Replying to [comment:5 mlennert]:
> > Can I close this ticket, as the specific bug is fixed, or does this
need a backport ?
>
> Not sure yet, I have to test releasebranch, because there were some
changes, reverts, and I can't do it right now.
I fixed it there as well. I think we should test it on Windows before we
close it.
#2727: wxGUI startup: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal
not in range(128)
--------------------------+-------------------------
Reporter: mlennert | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.0.2
Component: wxGUI | Version: svn-trunk
Resolution: | Keywords: encoding
CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------
Comment (by glynn):
Replying to [comment:2 mlennert]:
> It seems to me that we need to fundamentally rethink encoding handling,
but that up to now encoding issues have been 'fixed' one by one without
getting at the underlying weaknesses. But maybe I'm completely wrong in
that understanding...
When issues arise due to unicode strings being passed to functions
expecting byte string, the issue is normally fixed by adding an encode()
call in the caller. But to catch other such cases, it would help to also
modify the callee to explicitly check (e.g. via an assert statement) that
the value is a byte string rather than a unicode string.
An explicit check will catch cases where the passed-in string happens to
be convertible to ASCII. Relying upon Python raising an UnicodeError
doesn't help much when most of the strings being passed around are
convertible to ASCII.
Ideally, such checks should be added at the lowest point of the call
chain, in order to catch all possible routes to the point where the error
occurs.
#2727: wxGUI startup: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal
not in range(128)
--------------------------+-------------------------
Reporter: mlennert | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.0.3
Component: wxGUI | Version: svn-trunk
Resolution: | Keywords: encoding
CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------
Comment (by sebastic):
This issue has also been reported in [https://bugs.debian.org/806257
Debian Bug #806257], that user also has a French locale but a different
position in the error message:
{{{
'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in
range(128)
}}}
#2727: wxGUI startup: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal
not in range(128)
--------------------------+-------------------------
Reporter: mlennert | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.0.3
Component: wxGUI | Version: svn-trunk
Resolution: | Keywords: encoding
CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------
Comment (by mlennert):
Replying to [comment:10 sebastic]:
> This issue has also been reported in [https://bugs.debian.org/806257
Debian Bug #806257], that user also has a French locale but a different
position in the error message:
> {{{
> 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in
range(128)
> }}}
I think this is more related to #2827 which has been fixed now.
#2727: wxGUI startup: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal
not in range(128)
--------------------------+-------------------------
Reporter: mlennert | Owner: grass-dev@…
Type: defect | Status: closed
Priority: normal | Milestone: 7.0.3
Component: wxGUI | Version: svn-trunk
Resolution: fixed | Keywords: encoding
CPU: Unspecified | Platform: Unspecified
--------------------------+-------------------------
Comment (by mlennert):
Replying to [comment:11 mlennert]:
> Replying to [comment:10 sebastic]:
> > This issue has also been reported in [https://bugs.debian.org/806257
Debian Bug #806257], that user also has a French locale but a different
position in the error message:
> > {{{
> > 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in
range(128)
> > }}}
>
> I think this is more related to #2827 which has been fixed now.
