#3925: winGRASS 7.8.1dev: 'charmap' codec can't decode byte 0x9d - issue in vector
attribute data handling (e.g. opening attribute table, v.report, etc)
----------------------+------------------------------------
Reporter: hellik | Owner: grass-dev@…
Type: defect | Status: new
Priority: blocker | Milestone: 7.8.1
Component: Vector | Version: git-releasebranch78
Resolution: | Keywords: python3, py3, wingrass
CPU: x86-64 | Platform: MSWindows
----------------------+------------------------------------
Comment (by hellik):
now tested in a windows 10 box with a german locale:
{{{
GRASS Version: 7.8.1dev
Code revision: f5bfe545c
Build date: 2019-10-27
Build platform: x86_64-w64-mingw32
GDAL: 2.4.1
PROJ: 5.2.0
GEOS: 3.8.0
SQLite: 3.29.0
Python: 3.7.0
wxPython: 4.0.3
Platform: Windows-10-10.0.18362-SP0 (OSGeo4W)
}}}
download geonames data of an
[http://download.geonames.org/export/dump/AT.zip AT dump]
{{{
v.in.geonames input=D:\temp\geonames\AT\AT.txt output=at_geonames
Converting 51999 place names...
Scanne die Eingabe zur Ermittelung der Spaltentypen...
Number of columns: 19
Number of rows: 51999
WARNING: Spalte Nummer 13 <admin3code> ist als string definiert und hat
nur Integer-Werte.
WARNING: Spalte Nummer 14 <admin4code> ist als string definiert und hat
nur Integer-Werte.
Importiere Punkte...
Fülle Tabelle...
Erstelle Topologie für die Vektorkarte <at_geonames@data>...
Registriere Primitive...
GRASS_INFO_PROGRESS: 10000
GRASS_INFO_PROGRESS: 20000
GRASS_INFO_PROGRESS: 30000
GRASS_INFO_PROGRESS: 40000
GRASS_INFO_PROGRESS: 50000
}}}
clicking on a point with a german umlaut
{{{
east, north: 11.31700474110463, 47.23295282435024
at_geonames@data:
Type: Point
Id: 10136
Layer: 1
Category: 10136
Driver: sqlite
Database: D:\grassdata\loc_test_vingeonames\data\sqlite\sqlite.db
Table: at_geonames
Key_column: cat
Attributes:
cat: 10136
geonameid: 2762446
name: Vellenberg
asciiname: Vellenberg
latitude: 47.23333
longitude: 11.31667
featureclass: S
featurecode: FRM
countrycode: AT
admin1code: 07
admin2code: 703
admin3code: 70312
population: 0
gtopo30: 865
timezone: Europe/Vienna
modification: 2014-05-03
at_geonames@data:
Type: Point
Id: 25781
Layer: 1
Category: 25781
Driver: sqlite
Database: D:\grassdata\loc_test_vingeonames\data\sqlite\sqlite.db
Table: at_geonames
Key_column: cat
Attributes:
cat: 25781
geonameid: 2778215
name: Götznerberg
asciiname: Goetznerberg
latitude: 47.23333
longitude: 11.31667
featureclass: S
featurecode: FRM
countrycode: AT
admin1code: 07
admin2code: 703
admin3code: 70312
population: 0
gtopo30: 865
timezone: Europe/Vienna
modification: 2014-05-03
}}}
trying v.report
{{{
v.report map=at_geonames@data option=coor
Traceback (most recent call last):
File "C:\OSGEO4~1\apps\grass\grass78/scripts/v.report.py",
line 226, in <module>
main()
File "C:\OSGEO4~1\apps\grass\grass78/scripts/v.report.py",
line 108, in main
cols = decode(line).rstrip('\r\n').split('|')
File "C:\OSGEO4~1\apps\grass\grass78\etc\python\grass\scri
pt\utils.py", line 193, in decode
return bytes_.decode(enc)
File "C:\OSGEO4~1\apps\Python37\lib\encodings\cp1252.py",
line 15, in decode
return
codecs.charmap_decode(input,errors,decoding_table)
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d
in position 195: character maps to <undefined>
}}}
or trying to open the vector attribute table
{{{
Traceback (most recent call last):
File
"C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\lmgr\frame.py",
line 2060, in OnShowAttributeTable
selection=selection)
File "C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\dbmgr\ma
nager.py", line 112, in __init__
self.CreateDbMgrPage(parent=self, pageName='browse')
File
"C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\dbmgr\base.py",
line 811, in CreateDbMgrPage
parent=parent, parentDbMgrBase=self, onlyLayer=onlyLayer)
File
"C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\dbmgr\base.py",
line 1095, in __init__
self.AddLayer(layer)
File
"C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\dbmgr\base.py",
line 1138, in AddLayer
self.dbMgrData, layer, self.pages)
File
"C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\dbmgr\base.py",
line 113, in __init__
keyColumn = self.LoadData(layer)
File
"C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\dbmgr\base.py",
line 278, in LoadData
record = decode(outFile.readline().strip()).replace('\n',
'')
File "C:\OSGEO4~1\apps\grass\grass78\etc\python\grass\scri
pt\utils.py", line 193, in decode
return bytes_.decode(enc)
File "C:\OSGEO4~1\apps\Python37\lib\encodings\cp1252.py",
line 15, in decode
return codecs.charmap_decode(input,errors,decoding_table)
UnicodeDecodeError
:
'charmap' codec can't decode byte 0x9d in position 219:
character maps to <undefined>
}}}
opening the table freezes the attribute table window
it seems to be an encoding issue of attribute data handling
{{{
'charmap' codec can't decode byte 0x9d
}}}
--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3925#comment:2>
GRASS GIS <https://grass.osgeo.org>