#3220: WinGRASS not recognizing accented utf-8 (nor cp1252) attribute values
-------------------------+---------------------------------
Reporter: hellik | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.2.0
Component: Default | Version: svn-releasebranch72
Keywords: | CPU: Unspecified
Platform: MSWindows 8 |
-------------------------+---------------------------------
taken from the user ML:
https://lists.osgeo.org/pipermail/grass-user/2016-December/075682.html
{{{
I've got shape files with Swedish accented letters (ÄÖÅ) in the some of
the attribute
values. The Attributes are shwon as they should in the GUI. SQL
statements, however,
are not recognizing them. They're also messed up in the command output if
another
(not accented) values are queried.
I sat GRASS_DB_ENCODING to cp1252 firstly and it didn't work. Then I
converted the
dbf file into utf-8 and sat it as the value of the variable, to no avail.
I also
tried using the 'encoding' parameter in v.in.ogr in both cases, didn't
work.
I tried it on windows 8.1 and windows 10. The same is happening in both,
stable GRASS
7.0.5 and GRASS 7.2.0RC1.
The problem is only happening on Windows. Fedora and Mac OsX don't have
this issue
with the same shape files.
}}}
https://lists.osgeo.org/pipermail/grass-user/2016-December/075688.html
{{{
confirmed with
GRASS version: 7.3.svn
GRASS SVN revision: r70001
Build date: 2016-12-06
Build platform: x86_64-w64-mingw32
GDAL: 2.1.2
PROJ.4: 4.9.3
GEOS: 3.5.0
SQLite: 3.14.1
Python: 2.7.5
wxPython: 2.8.12.1
Platform: Windows-8-6.2.9200 (OSGeo4W)
}}}
{{{
and a test vector with following attributes
v.db.select map=test_points at data file=D:\temp\test_point.txt
cat|id|names
1|1|ÄÖÅ
2||Æ
3||Ø
4||Å,å,Æ,æ,Ø,ø
5||ø, Ø
6||Þ
7||Ð
8||Å
9||æ
}}}
{{{
d.vect map=test_points2 at data where="names = 'Å,å,Æ,æ,Ø,ø'" width=1
icon=basic/point size=10
doesn't show the selected point in the map display.
}}}
{{{
v.report map=test_points at data option=coor
cat|id|names|x|y|z
1|1|ÄÖÅ|1.37409120951759|47.039352838731|0.0
2||Æ|2.62326503635168|28.5515802015863|0.0
3||Ø|44.095836087244|57.2825782187707|0.0
4||Å,å,Æ,æ,Ø,ø|30.8545935228025|49.787535257766|0.0
5||ø, Ø|10.1183079973563|51.0367090846001|0.0
6||Þ|20.361533377396|52.0360481460674|0.0
8||Ã…|15.1491119517375|60.3621017805262|0.0
9||æ|-1.26290587954035|52.5879880709736|0.0
Traceback (most recent call last):
File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\gui\wxpython\gu
i_core\goutput.py", line 473, in OnCmdOutput
self.cmdOutput.AddStyledMessage(message, type)
File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\gui\wxpython\gu
i_core\goutput.py", line 772, in AddStyledMessage
self.AddTextWrapped(message, wrap=None)
File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\gui\wxpython\gu
i_core\goutput.py", line 721, in AddTextWrapped
txt = EncodeString(txt)
File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\gui\wxpython\co
re\gcmd.py", line 97, in EncodeString
return string.encode(_enc)
File "C:\OSGEO4~1\apps\Python27\lib\encodings\cp1252.py",
line 12, in encode
return codecs.charmap_encode(input,errors,encoding_table)
UnicodeDecodeError
:
'ascii' codec can't decode byte 0xc3 in position 3: ordinal
not in range(128)
}}}
--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3220>
GRASS GIS <https://grass.osgeo.org>