[GRASS-dev] g.mapsets: UnicodeDecodeError: 'utf8' codec can't decode bytes in position ...

GRASS GRASS developer
1

Hi,

I updated G6.4.svn (make distclean) and have g.mapsets failing (started with
no parameters) with

UnicodeDecodeError: 'utf8' codec can't decode bytes in position
84665-84666: invalid data
see
http://pastebin.com/BT1cHMJZ

I am using en_US.UTF-8 - no idea how to fix that problem...

Markus

2

Markus Neteler wrote:

Hi,

I updated G6.4.svn (make distclean) and have g.mapsets failing (started with
no parameters) with

UnicodeDecodeError: 'utf8' codec can't decode bytes in position
84665-84666: invalid data
see
http://pastebin.com/BT1cHMJZ

gui/wxpython/gui_modules/prompt.py:508

            fileHistory = codecs.open(os.path.join(env['GISDBASE'],
                                                   env['LOCATION_NAME'],
                                                   env['MAPSET'],
                                                   '.bash_history'),
                                      encoding = 'utf-8', mode = 'r')

This will result in the problem you describe if the .bash_history file
contains any non-UTF-8 data, which is a definite possibility
regardless of the user's locale settings.

I suggest adding errors='replace' or errors='ignore' when calling
codecs.open().

--
Glynn Clements <glynn@gclements.plus.com>

3

On Sat, May 14, 2011 at 4:16 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

Hi,

I updated G6.4.svn (make distclean) and have g.mapsets failing (started with
no parameters) with

UnicodeDecodeError: 'utf8' codec can't decode bytes in position
84665-84666: invalid data
see
http://pastebin.com/BT1cHMJZ

gui/wxpython/gui_modules/prompt.py:508

       fileHistory = codecs\.open\(os\.path\.join\(env\[&#39;GISDBASE&#39;\],
                                              env\[&#39;LOCATION\_NAME&#39;\],
                                              env\[&#39;MAPSET&#39;\],
                                              &#39;\.bash\_history&#39;\),
                                 encoding = &#39;utf\-8&#39;, mode = &#39;r&#39;\)

This will result in the problem you describe if the .bash_history file
contains any non-UTF-8 data, which is a definite possibility
regardless of the user's locale settings.

I suspect that I played in that mapset with Japanese locale
settings which left something in .bash_history.

I suggest adding errors='replace' or errors='ignore' when calling
codecs.open().

I just made a try with errors='replace' and it works. The GUI is back.

Markus