I found a quite striking issue when importing a simple vector file. It is a gpkg file with 30 points and only one field as attribute. When I import it into GRASS, either with v.in.ogr or v.import, I only get 28 points imported. No idea why. Meanwhile, ogrinfo -al reports 30 points, same as QGIS
Check if OGR layer contains polygons…
WARNING: Vector map already exists and will be overwritten
Creating attribute table for layer …
Importing 28 features (OGR layer )…
Building topology for vector map central@testing…
Registering primitives…
If I convert the gpkg to shapefile and import into GRASS, then all points are imported:
v.import input=central.shp layer=central output=central --overwrite
Check if OGR layer contains polygons…
Creating attribute table for layer …
Importing 30 features (OGR layer )…
Building topology for vector map central@testing…
Registering primitives…
Input <central.shp> successfully imported without reprojection
v.db.select -c central | wc -l
30
What could be the problem with gpkg and how to test further??
El lun., 8 oct. 2018 a las 21:24, Veronica Andreo (<veroandreo@gmail.com>) escribió:
Hello devs
I found a quite striking issue when importing a simple vector file. It is a gpkg file with 30 points and only one field as attribute. When I import it into GRASS, either with v.in.ogr or v.import, I only get 28 points imported. No idea why. Meanwhile, ogrinfo -al reports 30 points, same as QGIS
Check if OGR layer contains polygons…
WARNING: Vector map already exists and will be overwritten
Creating attribute table for layer …
Importing 28 features (OGR layer )…
Building topology for vector map central@testing…
Registering primitives…
If I convert the gpkg to shapefile and import into GRASS, then all points are imported:
v.import input=central.shp layer=central output=central --overwrite
Check if OGR layer contains polygons…
Creating attribute table for layer …
Importing 30 features (OGR layer )…
Building topology for vector map central@testing…
Registering primitives…
Input <central.shp> successfully imported without reprojection
v.db.select -c central | wc -l
30
What could be the problem with gpkg and how to test further??
The problem is that the bounding box included in the gpkg does not cover all points: these missing 2 points are outside the bbox embedded in the gpkg. As a safety measure, v.in.ogr applies a spatial filter using the input layer’s bbox to exclude corrupt features (previously observed in some shapefiles). This spatial filter is passed directly to OGR and as a consequence, OGR provides only 28 and not 30 points to v.in.ogr.
I am not sure what to do about this. Disable this safety check again?
Markus M
El lun., 8 oct. 2018 a las 21:24, Veronica Andreo (<veroandreo@gmail.com>) escribió:
Hello devs
I found a quite striking issue when importing a simple vector file. It is a gpkg file with 30 points and only one field as attribute. When I import it into GRASS, either with v.in.ogr or v.import, I only get 28 points imported. No idea why. Meanwhile, ogrinfo -al reports 30 points, same as QGIS
v.in.ogr input=central.gpkg layer=central output=central
Check if OGR layer contains polygons…
WARNING: Vector map already exists and will be overwritten
Creating attribute table for layer …
Importing 28 features (OGR layer )…
Building topology for vector map central@testing…
Registering primitives…
If I convert the gpkg to shapefile and import into GRASS, then all points are imported:
v.import input=central.shp layer=central output=central --overwrite
Check if OGR layer contains polygons…
Creating attribute table for layer …
Importing 30 features (OGR layer )…
Building topology for vector map central@testing…
Registering primitives…
Input <central.shp> successfully imported without reprojection
v.db.select -c central | wc -l
30
What could be the problem with gpkg and how to test further??
The problem is that the bounding box included in the gpkg does not cover all points: these missing 2 points are outside the bbox embedded in the gpkg. As a safety measure, v.in.ogr applies a spatial filter using the input layer’s bbox to exclude corrupt features (previously observed in some shapefiles). This spatial filter is passed directly to OGR and as a consequence, OGR provides only 28 and not 30 points to v.in.ogr.
And how is this bounding box created/defined, I was not aware of it.
I only converted from .kml to gpkg and then imported into GRASS.
I am not sure what to do about this. Disable this safety check again?
Maybe the bb safety check makes sense for features other than points, dunno. Others can for sure tell some use cases that I am not aware of. But I can say that it is pretty striking to see that some features are “eaten and lost” when importing into GRASS while ogr and others show everything.
What about a flag? Import all by default and the safety check with a flag? Or at least a note in the manual page of v.in.ogr
I am not sure what to do about this. Disable this safety check again?
Maybe the bb safety check makes sense for features other than points, dunno. Others can for sure tell some use cases that I am not aware of. But I can say that it is pretty striking to see that some features are "eaten and lost" when importing into GRASS while ogr and others show everything.
What about a flag? Import all by default and the safety check with a flag? Or at least a note in the manual page of v.in.ogr
My two cents' worth: from my perspective as a user, the main problem
is that it drops points without a warning (most of the time I am
blissfully unaware of the exact number of features that need import).
So throwing a warning should be considered, with maybe a flag as
suggested by Vero to void that check.
If I convert the gpkg to shapefile and import into GRASS, then all points are imported:
v.import input=central.shp layer=central output=central --overwrite
Check if OGR layer contains polygons…
Creating attribute table for layer …
Importing 30 features (OGR layer )…
Building topology for vector map central@testing…
Registering primitives…
Input <central.shp> successfully imported without reprojection
v.db.select -c central | wc -l
30
What could be the problem with gpkg and how to test further??
The problem is that the bounding box included in the gpkg does not cover all points: these missing 2 points are outside the bbox embedded in the gpkg. As a safety measure, v.in.ogr applies a spatial filter using the input layer’s bbox to exclude corrupt features (previously observed in some shapefiles). This spatial filter is passed directly to OGR and as a consequence, OGR provides only 28 and not 30 points to v.in.ogr.
And how is this bounding box created/defined, I was not aware of it.
It is created by the software that created that layer.
I only converted from .kml to gpkg and then imported into GRASS.
In this case something must have gone wrong when converting from kml to gpkg.
I am not sure what to do about this. Disable this safety check again?
Maybe the bb safety check makes sense for features other than points, dunno. Others can for sure tell some use cases that I am not aware of. But I can say that it is pretty striking to see that some features are “eaten and lost” when importing into GRASS while ogr and others show everything.
What about a flag? Import all by default and the safety check with a flag? Or at least a note in the manual page of v.in.ogr
I would rather disable this safety check if it does more harm than good.
If I convert the gpkg to shapefile and import into GRASS, then all points are imported:
v.import input=central.shp layer=central output=central --overwrite
Check if OGR layer contains polygons…
Creating attribute table for layer …
Importing 30 features (OGR layer )…
Building topology for vector map central@testing…
Registering primitives…
Input <central.shp> successfully imported without reprojection
v.db.select -c central | wc -l
30
What could be the problem with gpkg and how to test further??
The problem is that the bounding box included in the gpkg does not cover all points: these missing 2 points are outside the bbox embedded in the gpkg. As a safety measure, v.in.ogr applies a spatial filter using the input layer’s bbox to exclude corrupt features (previously observed in some shapefiles). This spatial filter is passed directly to OGR and as a consequence, OGR provides only 28 and not 30 points to v.in.ogr.
And how is this bounding box created/defined, I was not aware of it.
It is created by the software that created that layer.
I only converted from .kml to gpkg and then imported into GRASS.
In this case something must have gone wrong when converting from kml to gpkg.
I am not sure what to do about this. Disable this safety check again?
Maybe the bb safety check makes sense for features other than points, dunno. Others can for sure tell some use cases that I am not aware of. But I can say that it is pretty striking to see that some features are “eaten and lost” when importing into GRASS while ogr and others show everything.
What about a flag? Import all by default and the safety check with a flag? Or at least a note in the manual page of v.in.ogr
I would rather disable this safety check if it does more harm than good.
trešd., 2018. g. 10. okt., plkst. 16:04 — lietotājs Markus Metz
(<markus.metz.giswork@gmail.com>) rakstīja:
>> I am not sure what to do about this. Disable this safety check again?
>
>
> Maybe the bb safety check makes sense for features other than points, dunno. Others can for sure tell some use cases that I am not aware of. But I can say that it is pretty striking to see that some features are "eaten and lost" when importing into GRASS while ogr and others show everything.
>
> What about a flag? Import all by default and the safety check with a flag? Or at least a note in the manual page of v.in.ogr
I would rather disable this safety check if it does more harm than good.
I would vote for:
1) adding a flag to activate safety check;
2) disable safety check by default;
3) add a check "point in bbox" during import and print a warning if
any point of any geometry is outside of bbox (aka safety check without
action).
I am not sure what to do about this. Disable this safety check again?
Maybe the bb safety check makes sense for features other than points, dunno. Others can for sure tell some use cases that I am not aware of. But I can say that it is pretty striking to see that some features are “eaten and lost” when importing into GRASS while ogr and others show everything.
What about a flag? Import all by default and the safety check with a flag? Or at least a note in the manual page of v.in.ogr
I would rather disable this safety check if it does more harm than good.
I have disabled this safety check in trunk r73522.
I would vote for:
adding a flag to activate safety check;
disable safety check by default;
add a check “point in bbox” during import and print a warning if
any point of any geometry is outside of bbox (aka safety check without
action).
In this case there should also be a check “line in bbox” and “polygon in bbox”.
Right now we let OGR do all the spatial filtering which works just fine, often using features of the underlying vector format. I would prefer to leave this to OGR and not do this in GRASS.
