[GRASS-dev] [GRASS GIS] #2539: v.in.ogr: error importing shapefiles: ValueError : too many values to unpack

GRASS GIS GRASS GIS developer
,
1

#2539: v.in.ogr: error importing shapefiles: ValueError : too many values to
unpack
-------------------------------------+--------------------------------------
Reporter: richardc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: critical | Milestone: 7.0.0
Component: Vector | Version: svn-releasebranch70
Keywords: v.in.ogr vector import | Platform: Linux
      Cpu: x86-32 |
-------------------------------------+--------------------------------------
Probably related to https://trac.osgeo.org/grass/ticket/2537

I'm getting the following error on attempting to import a shape file,
without success, with v.in.ogr.

Layer Manager > File > Import vector data > Common import formats
[v.in.ogr] Browse to select file

From the command console:
{{{
Traceback (most recent call last):
   File "/usr/lib/python2.7/dist-
packages/wx-2.8-gtk2-unicode/wx/lib/filebrowsebutton.py",
line 140, in OnChanged

self.changeCallback(evt)
   File "/home/rcooper/grass70/grass-7.0.0svn/gui/wxpython/gu
i_core/gselect.py", line 1920, in OnUpdate

self._reloadLayers()
   File "/home/rcooper/grass70/grass-7.0.0svn/gui/wxpython/gu
i_core/gselect.py", line 1945, in _reloadLayers

layerName, featureType, projection = map(lambda x:
x.strip(), line.split(','))
ValueError
:
too many values to unpack
{{{

And after clicking Import, I receive the following popup message:
No layers selected. Operation canceled

The above relates to 7.0.0svn:
{{{
System Info
GRASS version: 7.0.0svn
GRASS SVN Revision: 63929
Build Date: 2015-01-02
Build Platform: i686-pc-linux-gnu
GDAL/OGR: 1.11.1
PROJ.4: 4.9.0
GEOS: 3.4.2
SQLite: 3.7.9
Python: 2.7.3
wxPython: 2.8.12.1
Platform: Linux-3.2.0-31-generic-pae-i686-with-LinuxMint-13-maya
}}}

I've tried with a couple of shapefiles.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2539&gt;
GRASS GIS <http://grass.osgeo.org>

2

#2539: v.in.ogr: error importing shapefiles: ValueError : too many values to
unpack
----------------------------------------+-----------------------------------
Reporter: richardc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-releasebranch70
Keywords: wxgui vector import wizard | Platform: Linux
      Cpu: x86-32 |
----------------------------------------+-----------------------------------
Changes (by mlennert):

  * keywords: v.in.ogr vector import => wxgui vector import wizard
  * priority: critical => blocker
  * component: Vector => wxGUI

Comment:

I can confirm this error on a freshly checkouted and compiled
grass70release tree.

Bumping this up to blocker as this is fundamental functionality of the
GUI.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2539#comment:1&gt;
GRASS GIS <http://grass.osgeo.org>

3

#2539: v.in.ogr: error importing shapefiles: ValueError : too many values to
unpack
----------------------------------------+-----------------------------------
Reporter: richardc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-releasebranch70
Keywords: wxgui vector import wizard | Platform: Linux
      Cpu: x86-32 |
----------------------------------------+-----------------------------------

Comment(by hellik):

Replying to [comment:1 mlennert]:
> I can confirm this error on a freshly checkouted and compiled
grass70release tree.
>
> Bumping this up to blocker as this is fundamental functionality of the
GUI.

tested and ''not'' confirmed by

{{{
System Info
GRASS Version: 7.1.svn
GRASS SVN Revision: 64069
Erstellungsdatum: 2015-01-12
Build Platform: i686-pc-mingw32
GDAL/OGR: 1.11.1
PROJ.4: 4.8.0
GEOS: 3.4.2
SQLite: 3.7.17
Python: 2.7.4
wxPython: 2.8.12.1
Platform: Windows-7-6.1.7601-SP1 (OSGeo4W)
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2539#comment:2&gt;
GRASS GIS <http://grass.osgeo.org>

4

#2539: v.in.ogr: error importing shapefiles: ValueError : too many values to
unpack
----------------------------------------+-----------------------------------
Reporter: richardc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-releasebranch70
Keywords: wxgui vector import wizard | Platform: Linux
      Cpu: x86-32 |
----------------------------------------+-----------------------------------

Comment(by martinl):

I cannot reproduce this bug. Please can you check it again with the fresh
installation?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2539#comment:3&gt;
GRASS GIS <http://grass.osgeo.org>

5

#2539: v.in.ogr: error importing shapefiles: ValueError : too many values to
unpack
----------------------------------------+-----------------------------------
Reporter: richardc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-releasebranch70
Keywords: wxgui vector import wizard | Platform: Linux
      Cpu: x86-32 |
----------------------------------------+-----------------------------------

Comment(by mlennert):

Replying to [comment:3 martinl]:
> I cannot reproduce this bug. Please can you check it again with the
fresh installation?

I have trouble finding a clear logic in all this as sometimes it seems to
work and sometimes not. But I think I found a candidate of what might
explain it
[http://trac.osgeo.org/grass/browser/grass/trunk/vector/v.external/list.c#L326
1], but only with gdal < 1.11 :

{{{
312 #if GDAL_VERSION_NUM >= 1110000
313 for (igeom = 0; igeom <
OGR_FD_GetGeomFieldCount(Ogr_featuredefn); igeom++) {
314 Ogr_geomdefn =
OGR_FD_GetGeomFieldDefn(Ogr_featuredefn, igeom);
315 if (!Ogr_geomdefn) {
316 G_warning(_("Invalid geometry column %d"),
igeom);
317 continue;
318 }
319
320 Ogr_geom_type =
OGR_GFld_GetType(Ogr_geomdefn);
321 fprintf(fd, "%s,%s,%d,%s\n", layer_name,
322
feature_type(OGRGeometryTypeToName(Ogr_geom_type)),
323 proj_same,
OGR_GFld_GetNameRef(Ogr_geomdefn));
324 }
325 #else
326 fprintf(fd, "%s,%s,%d,\n", layer_name,
327
feature_type(OGRGeometryTypeToName(Ogr_geom_type)),
328 proj_same);
329 #endif
}}}

Line 326 reads:

{{{
fprintf(fd, "%s,%s,%d,\n"
}}}

should the last comma really be there ? Erasing it away seems to solve the
problem for me.

However, this would not explain the problem for RichardC as he seems to be
working with gdal 1.11...

Moritz

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2539#comment:4&gt;
GRASS GIS <http://grass.osgeo.org>

6

#2539: v.in.ogr: error importing shapefiles: ValueError : too many values to
unpack
----------------------------------------+-----------------------------------
Reporter: richardc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-releasebranch70
Keywords: wxgui vector import wizard | Platform: Linux
      Cpu: x86-32 |
----------------------------------------+-----------------------------------

Comment(by martinl):

Replying to [comment:4 mlennert]:
> Replying to [comment:3 martinl]:
> > I cannot reproduce this bug. Please can you check it again with the
fresh installation?
>
> I have trouble finding a clear logic in all this as sometimes it seems
to work and sometimes not.

I updated `v.external` to report also geometry columns, see (1). The bug
was related to PostgreSQL format for which `v.external` is not using GDAL
library by default, but it's own implementation. Now it should be fixed.

(1) http://trac.osgeo.org/gdal/wiki/rfc41_multiple_geometry_fields

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2539#comment:5&gt;
GRASS GIS <http://grass.osgeo.org>

7

#2539: v.in.ogr: error importing shapefiles: ValueError : too many values to
unpack
----------------------------------------+-----------------------------------
Reporter: richardc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-releasebranch70
Keywords: wxgui vector import wizard | Platform: Linux
      Cpu: x86-32 |
----------------------------------------+-----------------------------------

Comment(by martinl):

Replying to [comment:4 mlennert]:
> should the last comma really be there ? Erasing it away seems to solve
the problem for me.

sorry, what kind of problems?

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2539#comment:6&gt;
GRASS GIS <http://grass.osgeo.org>

8

#2539: v.in.ogr: error importing shapefiles: ValueError : too many values to
unpack
----------------------------------------+-----------------------------------
Reporter: richardc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-releasebranch70
Keywords: wxgui vector import wizard | Platform: Linux
      Cpu: x86-32 |
----------------------------------------+-----------------------------------

Comment(by mlennert):

Replying to [comment:6 martinl]:
> Replying to [comment:4 mlennert]:
> > should the last comma really be there ? Erasing it away seems to solve
the problem for me.
>
> sorry, what kind of problems?

The "ValueError: too many values to unpack" error when trying to use the
vector import wizard to import a shapefile. This is a different bug (but
probably of similar origin) than #2537.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2539#comment:7&gt;
GRASS GIS <http://grass.osgeo.org>

9

#2539: v.in.ogr: error importing shapefiles: ValueError : too many values to
unpack
----------------------------------------+-----------------------------------
Reporter: richardc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-releasebranch70
Keywords: wxgui vector import wizard | Platform: Linux
      Cpu: x86-32 |
----------------------------------------+-----------------------------------

Comment(by martinl):

Replying to [comment:7 mlennert]:

> The "ValueError: too many values to unpack" error when trying to use the
vector import wizard to import a shapefile. This is a different bug (but
probably of similar origin) than #2537.

As I already wrote, I cannot reproduce this bug. Are you sure that you
have a fresh installation?

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2539#comment:8&gt;
GRASS GIS <http://grass.osgeo.org>

10

#2539: v.in.ogr: error importing shapefiles: ValueError : too many values to
unpack
----------------------------------------+-----------------------------------
Reporter: richardc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-releasebranch70
Keywords: wxgui vector import wizard | Platform: Linux
      Cpu: x86-32 |
----------------------------------------+-----------------------------------

Comment(by martinl):

Replying to [comment:4 mlennert]:

> However, this would not explain the problem for RichardC as he seems to
be working with gdal 1.11...

I have tested GRASS with GDAL 1.11, no problem.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2539#comment:9&gt;
GRASS GIS <http://grass.osgeo.org>

11

#2539: v.in.ogr: error importing shapefiles: ValueError : too many values to
unpack
----------------------------------------+-----------------------------------
Reporter: richardc | Owner: grass-dev@…
     Type: defect | Status: new
Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-releasebranch70
Keywords: wxgui vector import wizard | Platform: Linux
      Cpu: x86-32 |
----------------------------------------+-----------------------------------

Comment(by mlennert):

I'm really losing the little that is left of my hair over this... :wink:

I did a completely fresh checkout of relbr70 this morning and could
confirm the issue. Now I did the same and the issue seems to have gone
away. IIUC, the issue was fixed with r64041, changing

{{{
layerName, featureType, projection = map(lambda x: x.strip(),
line.split(','))
}}}

to

{{{
layerName, featureType, projection, geometryColumn = map(lambda x:
x.strip(), line.split(','))
}}}

in gui/wxpython/gui_core/gselect.py, thus solving the issue of only three
variables, but four values.

I don't understand why I still saw this error this morning, though.

I'll check out, recompile and test tomorrow on other machines running with
gdal 1.10. I'll keep it open until then.

P.S. Could it be that certain gui files are not updated with 'make
distclean; svn up; configure; make' ? I shouldn't have to do fresh
checkouts everytime...

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2539#comment:10&gt;
GRASS GIS <http://grass.osgeo.org>

12

#2539: v.in.ogr: error importing shapefiles: ValueError : too many values to
unpack
-----------------------+----------------------------------------------------
  Reporter: richardc | Owner: grass-dev@…
      Type: defect | Status: closed
  Priority: blocker | Milestone: 7.0.0
Component: wxGUI | Version: svn-releasebranch70
Resolution: fixed | Keywords: wxgui vector import wizard
  Platform: Linux | Cpu: x86-32
-----------------------+----------------------------------------------------
Changes (by mlennert):

  * status: new => closed
  * resolution: => fixed

Comment:

Replying to [comment:10 mlennert]:
> I'm really losing the little that is left of my hair over this... :wink:
>
> I did a completely fresh checkout of relbr70 this morning and could
confirm the issue. Now I did the same and the issue seems to have gone
away. IIUC, the issue was fixed with r64041, changing
>
>
> {{{
> layerName, featureType, projection = map(lambda x: x.strip(),
line.split(','))
> }}}
>
> to
>
>
> {{{
> layerName, featureType, projection, geometryColumn = map(lambda x:
x.strip(), line.split(','))
> }}}
>
> in gui/wxpython/gui_core/gselect.py, thus solving the issue of only
three variables, but four values.
>
> I don't understand why I still saw this error this morning, though.
>
> I'll check out, recompile and test tomorrow on other machines running
with gdal 1.10. I'll keep it open until then.
>

Just did that with relbr7 and everything seems to work, now. Thanks for
the fixes.

Closing the ticket.

Moritz

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2539#comment:11&gt;
GRASS GIS <http://grass.osgeo.org>