[GRASS-dev] [GRASS GIS] #2300: v.out.ogr: spatialite support not working

#2300: v.out.ogr: spatialite support not working
-----------------------------------+----------------------------------------
Reporter: hamish | Owner: grass-dev@…
     Type: defect | Status: new
Priority: critical | Milestone: 6.4.4
Component: Vector | Version: svn-trunk
Keywords: v.out.ogr, spatialite | Platform: Linux
      Cpu: x86-64 |
-----------------------------------+----------------------------------------
Hi, as noted on the grass-users ML,
  http://lists.osgeo.org/pipermail/grass-user/2014-May/070328.html

I'm not able to get Spatialite export working from v.out.ogr. Seen on both
GRASS 6.x and latest trunk for debian/wheezy.

Loading the mapset into the qgis grass toolbox on the same machine and
saving from there works, so I think the system libraries are all ok (gdal
1.9.0).

for my test I'm trying the usgsgages vector map from the North Carolina
dataset. I've tried roadsmajor type=line and census_wake2000 type=area and
they give the same trouble.

{{{
v.out.ogr in=usgsgages dsn=usgsgages.sqlite \
    format=SQLite type=point dsco='SPATIALITE=yes'
}}}

the error, reported for each point is:

{{{
ERROR 1: sqlite3_step() failed:
   usgsgages.GEOMETRY violates Geometry constraint [geom-type or SRID not
allowed] (19)
}}}

In trunk it seems to be happening on line 128 of v.in.ogr's export_lines.c
(~788 in main.c for devbr6):
{{{
   OGR_L_CreateFeature(Ogr_layer, Ogr_feature);
}}}

api ref:
http://www.gdal.org/ogr/ogr__api_8h.html#a301d319111285a47fe6cda6e079214f8

MarkusN pulled up some hints from the gdal ML:
  http://lists.osgeo.org/pipermail/gdal-dev/2013-May/036148.html [[BR]]
  http://lists.osgeo.org/pipermail/gdal-dev/2013-May/036152.html

where Even wrote:
> The driver takes responsibility of assigning the SRID automatically from
> the layer SRS. So the error is likely due to an attempt of inserting a
> geometry whose type doesn't match the layer geometry type. Spatialite
> is really strict on that: POLYGON != MULTIPOLYGON, and (perhaps I'm not
> sure) 2D != 2.5D

In the gdal-dev thread the solution was to pick the correct one
of OGR_G_SetPoint() vs. OGR_G_SetPoint_2D(), maybe that helps here?

the code seems suitably pointy, even if it is embedded in the "lines" part
of the code, but I wonder if 2D vs 3D definition is causing trouble?
Converting usgsgages to a 3D points map didn't help.

{{{
     else if ((type == GV_POINT) ||
              ((type == GV_CENTROID) && (otype & GV_CENTROID))) {

         Ogr_geometry = OGR_G_CreateGeometry(wkbPoint);
         OGR_G_AddPoint(Ogr_geometry, Points->x[0], Points->y[0],
                        Points->z[0]);
     }
}}}

from the partially created sqlite file created by v.out.ogr here's the
creation log:
{{{
$ echo "SELECT * from spatialite_history;" | sqlite3 usgsgages4.sqlite
1|spatial_ref_sys||table successfully created|2014-05-19
23:14:31|1|3.7.13|3.0.0-beta
2|geometry_columns||table successfully created|2014-05-19
23:14:31|2|3.7.13|3.0.0-beta
3|spatial_ref_sys||table successfully populated|2014-05-19
23:14:32|3|3.7.13|3.0.0-beta
4|usgsgages|GEOMETRY|Geometry [POINT,XY,SRID=40004] successfully
created|2014-05-19 23:14:33|3.7.13|3.0.0-beta
5|usgsgages|GEOMETRY|R*Tree Spatial Index successfully created|2014-05-19
23:14:34|3.7.13|3.0.0-beta
}}}

One thing I notice is that SRID 40004 is a custom one added at the
end of the spatial_ref_sys table, after four other custom ones from
Italy. I'm not sure if that is relevant or not.

thanks,
Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2300&gt;
GRASS GIS <http://grass.osgeo.org>

#2300: v.out.ogr: spatialite support not working
-----------------------------------+----------------------------------------
Reporter: hamish | Owner: grass-dev@…
     Type: defect | Status: new
Priority: critical | Milestone: 6.4.4
Component: Vector | Version: svn-trunk
Keywords: v.out.ogr, spatialite | Platform: Linux
      Cpu: x86-64 |
-----------------------------------+----------------------------------------

Comment(by mlennert):

See also Debian bug 746566: [https://bugs.debian.org/cgi-
bin/bugreport.cgi?bug=746566 gdal-bin: ogr2ogr fails to create SQLite db
using SPATIALITE=YES].

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2300#comment:1&gt;
GRASS GIS <http://grass.osgeo.org>

#2300: v.out.ogr: spatialite support not working
-----------------------------------+----------------------------------------
Reporter: hamish | Owner: grass-dev@…
     Type: defect | Status: new
Priority: critical | Milestone: 6.4.4
Component: Vector | Version: svn-trunk
Keywords: v.out.ogr, spatialite | Platform: Linux
      Cpu: x86-64 |
-----------------------------------+----------------------------------------

Comment(by hamish):

Replying to [comment:1 mlennert]:
> See also Debian bug 746566: [https://bugs.debian.org/cgi-
bin/bugreport.cgi?bug=746566 gdal-bin: ogr2ogr fails to create SQLite db
using SPATIALITE=YES].

Hi,

I don't think the two are related, since the export fails with gdal 1.9.0
and qgis with the same libraries works fine. I more suspect the 2D vs. 3D
geometry struct issue.

thanks,
Hamish

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2300#comment:2&gt;
GRASS GIS <http://grass.osgeo.org>

#2300: v.out.ogr: spatialite support not working
-----------------------------------+----------------------------------------
Reporter: hamish | Owner: grass-dev@…
     Type: defect | Status: new
Priority: critical | Milestone: 6.4.4
Component: Vector | Version: svn-trunk
Keywords: v.out.ogr, spatialite | Platform: Linux
      Cpu: x86-64 |
-----------------------------------+----------------------------------------

Comment(by hamish):

here is another possibility,

  `"When creating a spatial table and recovering the geometry using the
RecoverGeometryColumn function if you don't specify the geometry column as
lower case you end up getting a 'violates Geometry constraint [geom-type
or SRID not allowed]' error."`

http://stackoverflow.com/questions/15203403/spatialite-v4-0-0
-recovergeometrycolumn-violates-geometry-constraint-geom-type

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2300#comment:3&gt;
GRASS GIS <http://grass.osgeo.org>

#2300: v.out.ogr: spatialite support not working
-----------------------------------+----------------------------------------
Reporter: hamish | Owner: grass-dev@…
     Type: defect | Status: new
Priority: critical | Milestone: 6.4.4
Component: Vector | Version: svn-trunk
Keywords: v.out.ogr, spatialite | Platform: Linux
      Cpu: x86-64 |
-----------------------------------+----------------------------------------

Comment(by hamish):

Replying to [ticket:2300 hamish]:
> In the gdal-dev thread the solution was to pick the correct one
> of OGR_G_SetPoint() vs. OGR_G_SetPoint_2D(), maybe that helps here?
>
> the code seems suitably pointy, even if it is embedded in the "lines"
> part of the code, but I wonder if 2D vs 3D definition is causing
trouble?

right, that's it. The following patch vs. devbr6 makes it work:

{{{
Index: main.c

--- main.c (revision 59151)
+++ main.c (working copy)
@@ -753,9 +753,13 @@
             }
             else if ((type == GV_POINT) ||
                      ((type == GV_CENTROID) && (otype & GV_CENTROID))) {
+
                 Ogr_geometry = OGR_G_CreateGeometry(wkbPoint);
- OGR_G_AddPoint(Ogr_geometry, Points->x[0], Points->y[0],
- Points->z[0]);
+ if (Vect_is_3d(&In))
+ OGR_G_AddPoint(Ogr_geometry, Points->x[0],
Points->y[0],
+ Points->z[0]);
+ else
+ OGR_G_AddPoint_2D(Ogr_geometry, Points->x[0],
Points->y[0]);
             }
             else { /* GV_LINE or GV_BOUNDARY */

}}}

but the v.out.ogr z-flag logic might require some modifications to that.
(the force-2D flag in trunk makes a bit more sense, but also would need a
different check for the flag)

> Converting usgsgages to a 3D points map didn't help.

but I don't think I used the -z flag... ? All ways need to be tested, and
other geometry types also need to receive the same 2D vs 3D treatment I
guess.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2300#comment:4&gt;
GRASS GIS <http://grass.osgeo.org>

#2300: v.out.ogr: spatialite support not working
-----------------------+-----------------------------------
  Reporter: hamish | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: critical | Milestone: 6.4.4
Component: Vector | Version: svn-trunk
Resolution: | Keywords: v.out.ogr, spatialite
       CPU: x86-64 | Platform: Linux
-----------------------+-----------------------------------

Comment (by neteler):

It also fails in GRASS 7:

{{{
GRASS 7.0.1svn (nc_spm_08_grass7):~ > v.out.ogr input=usgsgages
output=usgsgages.sqlite format=SQLite type=point dsco='SPATIALITE=yes'
Exporting 433 features...
ERROR 1: sqlite3_step() failed:
   usgsgages.GEOMETRY violates Geometry constraint [geom-type or SRID not
allowed] (19)
...
ERROR 1: sqlite3_step() failed:
   usgsgages.GEOMETRY violates Geometry constraint [geom-type or SRID not
allowed] (19)
  100%
ERROR 1: sqlite3_step() failed:
   usgsgages.GEOMETRY violates Geometry constraint [geom-type or SRID not
allowed] (19)
ERROR 1: COMMIT transaction failed: cannot commit - no transaction is
active
v.out.ogr complete. 433 features (Point type) written to <usgsgages>
(SQLite format).
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2300#comment:5&gt;
GRASS GIS <http://grass.osgeo.org>