[GRASS-dev] [GRASS GIS] #3781: v.in.geonames - fails in winGRASS7.7 (python3)

GDAL · March 6, 2019, 9:24pm

#3781: v.in.geonames - fails in winGRASS7.7 (python3)
--------------------------+-------------------------
Reporter: hellik | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.8.0
Component: Python | Version: svn-trunk
Keywords: python3, py3 | CPU: x86-64
Platform: MSWindows |
--------------------------+-------------------------
tested with

{{{
System Info
GRASS version: 7.7.svn
GRASS SVN revision: r74150M
Build date: 2019-03-05
Build platform: x86_64-w64-mingw32
GDAL: 2.4.0
PROJ.4: 5.2.0
GEOS: 3.7.0
SQLite: 3.26.0
Python: 3.7.0
wxPython: 4.0.3
Platform: Windows-10-10.0.17763-SP0 (OSGeo4W)
}}}

v.in.geonames fails with

{{{
v.in.geonames input=D:\data\private\presentation\lernwerkstatt\AT\AT.txt
output=at_out
Traceback (most recent call last):
   File
"C:\OSGEO4~1\apps\grass\grass77/scripts/v.in.geonames.py",
line 150, in <module>
     main()
   File
"C:\OSGEO4~1\apps\grass\grass77/scripts/v.in.geonames.py",
line 67, in main
     num_places = sum(1 for _ in f)
   File
"C:\OSGEO4~1\apps\grass\grass77/scripts/v.in.geonames.py",
line 67, in <genexpr>
     num_places = sum(1 for _ in f)
   File "C:\OSGEO4~1\apps\Python37\lib\encodings\cp1252.py",
line 23, in decode
     return
codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d
in position 3815: character maps to <undefined>
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3781>
GRASS GIS <https://grass.osgeo.org>

GDAL · March 6, 2019, 9:26pm

#3781: v.in.geonames - fails in winGRASS7.7 (python3)
---------------------+--------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.8.0
Component: Python | Version: svn-trunk
Resolution: | Keywords: python3, py3
       CPU: x86-64 | Platform: MSWindows
---------------------+--------------------------

Comment (by hellik):

Replying to [ticket:3781 hellik]:
> tested with
>
> {{{
> System Info
> GRASS version: 7.7.svn
> GRASS SVN revision: r74150M
> Build date: 2019-03-05
> Build platform: x86_64-w64-mingw32
> GDAL: 2.4.0
> PROJ.4: 5.2.0
> GEOS: 3.7.0
> SQLite: 3.26.0
> Python: 3.7.0
> wxPython: 4.0.3
> Platform: Windows-10-10.0.17763-SP0 (OSGeo4W)
> }}}
>
> v.in.geonames fails with
>
> {{{
> v.in.geonames input=D:\data\private\presentation\lernwerkstatt\AT\AT.txt
output=at_out
> Traceback (most recent call last):
> File
> "C:\OSGEO4~1\apps\grass\grass77/scripts/v.in.geonames.py",
> line 150, in <module>
> main()
> File
> "C:\OSGEO4~1\apps\grass\grass77/scripts/v.in.geonames.py",
> line 67, in main
> num_places = sum(1 for _ in f)
> File
> "C:\OSGEO4~1\apps\grass\grass77/scripts/v.in.geonames.py",
> line 67, in <genexpr>
> num_places = sum(1 for _ in f)
> File "C:\OSGEO4~1\apps\Python37\lib\encodings\cp1252.py",
> line 23, in decode
> return
> codecs.charmap_decode(input,self.errors,decoding_table)[0]
> UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d
> in position 3815: character maps to <undefined>
> }}}

same data set works with

{{{
GRASS version: 7.6.0
GRASS SVN revision: r73958
Build date: 2019-01-17
Build platform: x86_64-w64-mingw32
GDAL: 2.4.0
PROJ.4: 5.2.0
GEOS: 3.7.0
SQLite: 3.26.0
Python: 2.7.14
wxPython: 2.8.12.1
Platform: Windows-10-10.0.17763 (OSGeo4W)
}}}

{{{
v.in.geonames input=D:\data\private\presentation\lernwerkstatt\AT\AT.txt
output=at2
Converting 51995 place names...
Scanning input for column types...
Number of columns: 19
Number of rows: 51995
WARNING: Column number 13 <admin3code> defined as string has only integer
values
WARNING: Column number 14 <admin4code> defined as string has only integer
values
Importing points...
Populating table...
Building topology for vector map <at2@data>...
Registering primitives...
(Wed Mar 06 22:25:18 2019) Command finished (9 sec)
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3781#comment:1>
GRASS GIS <https://grass.osgeo.org>

GDAL · May 15, 2019, 5:37pm

#3781: v.in.geonames - fails in winGRASS7.7 (python3)
---------------------+--------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.8.0
Component: Python | Version: svn-trunk
Resolution: | Keywords: python3, py3
       CPU: x86-64 | Platform: MSWindows
---------------------+--------------------------

Comment (by annakrat):

In [changeset:"74481" 74481]:
{{{
#!CommitTicketReference repository="" revision="74481"
v.in.geonames: read as utf8, see #3781
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3781#comment:2>
GRASS GIS <https://grass.osgeo.org>

GDAL · September 6, 2019, 8:21pm

#3781: v.in.geonames - fails in winGRASS7.7 (python3)
---------------------+---------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.8.0
Component: Python | Version: git-releasebranch78
Resolution: | Keywords: python3, py3
       CPU: x86-64 | Platform: MSWindows
---------------------+---------------------------------
Changes (by neteler):

* version: svn-trunk => git-releasebranch78

Comment:

Tested on Linux:

{{{
wget http://download.geonames.org/export/dump/AT.zip
unzip AT.zip

v.in.geonames input=AT.txt output=at_out
Converting 52003 place names...
Scanning input for column types...
Number of columns: 19
Number of rows: 52003
WARNING: Column number 13 <admin3code> defined as string has only integer
          values
WARNING: Column number 14 <admin4code> defined as string has only integer
          values
Importing points...
  100%
Populating table...
Building topology for vector map <at_out@user1>...
Registering primitives...
}}}

Still an issue?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3781#comment:3>
GRASS GIS <https://grass.osgeo.org>

GDAL · September 7, 2019, 2:30am

#3781: v.in.geonames - fails in winGRASS7.7 (python3)
---------------------+---------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.8.0
Component: Python | Version: git-releasebranch78
Resolution: | Keywords: python3, py3
       CPU: x86-64 | Platform: MSWindows
---------------------+---------------------------------

Comment (by annakrat):

The issue was on Windows, so someone should confirm the fix worked.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3781#comment:4>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 23, 2019, 6:51am

#3781: v.in.geonames - fails in winGRASS7.7 (python3)
---------------------+---------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.8.1
Component: Python | Version: git-releasebranch78
Resolution: | Keywords: python3, py3
       CPU: x86-64 | Platform: MSWindows
---------------------+---------------------------------

Comment (by neteler):

Ping Helli, still an issue?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3781#comment:6>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 23, 2019, 9:09pm

#3781: v.in.geonames - fails in winGRASS7.7 (python3)
---------------------+---------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone: 7.8.1
Component: Python | Version: git-releasebranch78
Resolution: | Keywords: python3, py3
       CPU: x86-64 | Platform: MSWindows
---------------------+---------------------------------

Comment (by hellik):

tested with

{{{
GRASS Version: 7.8.1dev
Code revision: d1c4ad132
Build date: 2019-10-22
Build platform: x86_64-w64-mingw32
GDAL: 2.4.1
PROJ: 5.2.0
GEOS: 3.8.0
SQLite: 3.29.0
Python: 3.7.0
wxPython: 4.0.3
Platform: Windows-10-10.0.18362-SP0 (OSGeo4W)
}}}

{{{
v.in.geonames --verbose input=D:\temp\geonames\AT\AT.txt output=at_out
Converting 51999 place names...
Using native format
Scanne die Eingabe zur Ermittelung der Spaltentypen...
Maximale Länge der Inputzeile: 2204
Number of columns: 19
Number of rows: 51999
Column: 1 type: integer
Column: 2 type: string length: 79
Column: 3 type: string length: 79
Column: 4 type: string length: 2089
Column: 5 type: double
Column: 6 type: double
Column: 7 type: string length: 1
Column: 8 type: string length: 5
Column: 9 type: string length: 2
Column: 10 type: string length: 8
Column: 11 type: string length: 2
Column: 12 type: string length: 2
Column: 13 type: integer
Column: 14 type: integer
Column: 15 type: integer
Column: 16 type: integer
Column: 17 type: integer
Column: 18 type: string length: 13
Column: 19 type: string length: 10
WARNING: Spalte Nummer 13 <admin3code> ist als string definiert und hat
nur Integer-Werte.
WARNING: Spalte Nummer 14 <admin4code> ist als string definiert und hat
nur Integer-Werte.
Importiere Punkte...
Fülle Tabelle...
Erstelle Topologie für die Vektorkarte <at_out@data>...
Registriere Primitive...
GRASS_INFO_PROGRESS: 10000
GRASS_INFO_PROGRESS: 20000
GRASS_INFO_PROGRESS: 30000
GRASS_INFO_PROGRESS: 40000
GRASS_INFO_PROGRESS: 50000
51999 primitives registered
51999 vertices registered
Die Topologie wurde erstellt.
Anzahl der Knoten: 0
Anzahl der Primitive: 51999
Anzahl der Punkte: 51999
Anzahl der Linien: 0
Anzahl der Grenzen: 0
Anzahl der Zentroide: 0
Anzahl der Flächen: 0
Anzahl der Inseln: 0
(Wed Oct 23 23:07:58 2019) Befehl ausgeführt (12 Sek)
}}}

seems to work now, closing ticket

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3781#comment:7>
GRASS GIS <https://grass.osgeo.org>

GDAL · October 23, 2019, 9:10pm

#3781: v.in.geonames - fails in winGRASS7.7 (python3)
---------------------+---------------------------------
  Reporter: hellik | Owner: grass-dev@…
      Type: defect | Status: closed
  Priority: normal | Milestone: 7.8.1
Component: Python | Version: git-releasebranch78
Resolution: fixed | Keywords: python3, py3
       CPU: x86-64 | Platform: MSWindows
---------------------+---------------------------------
Changes (by hellik):

* status: new => closed
* resolution: => fixed

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3781#comment:8>
GRASS GIS <https://grass.osgeo.org>