v.in.osm input=Downloads/berlin-latest.osm.pbf table=lines
type=point,line output=roads --o
WARNING: Width for column osm_id set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Width for column name set to 255 (was not specified by OGR), some
strings may be truncated!
WARNING: Width for column highway set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Width for column waterway set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Width for column aerialway set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Width for column barrier set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Width for column man_made set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Width for column other_tags set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Vector map <roads> already exists and will be overwritten
WARNING: No data base element files found
v.in.osm input=Downloads/berlin-latest.osm.pbf table=lines
type=point,line output=roads --o
WARNING: Width for column osm_id set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Width for column name set to 255 (was not specified by OGR), some
strings may be truncated!
WARNING: Width for column highway set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Width for column waterway set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Width for column aerialway set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Width for column barrier set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Width for column man_made set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Width for column other_tags set to 255 (was not specified by OGR),
some strings may be truncated!
WARNING: Vector map already exists and will be overwritten
WARNING: No data base element files found
v.in.ogr input=~/Downloads/berlin-latest.osm.pbf type=point,line output=roads
...
Number of nodes: 177060
Number of primitives: 743673
Number of points: 550356
Number of lines: 193317
Number of boundaries: 0
Number of centroids: 0
Number of areas: 0
Number of isles: 0
Nice!
GRASS 7.2.2svn (latlong_wgs84):~ > v.db.connect -p roads
Vector map <roads> is connected by:
layer <1/points> table <roads_1> in database
</home/mneteler/grassdata/latlong_wgs84/osm/sqlite/sqlite.db> through
driver <sqlite> with key <cat>
layer <2/lines> table <roads_2> in database
</home/mneteler/grassdata/latlong_wgs84/osm/sqlite/sqlite.db> through
driver <sqlite> with key <cat>
layer <3/multilinestrings> table <roads_3> in database
</home/mneteler/grassdata/latlong_wgs84/osm/sqlite/sqlite.db> through
driver <sqlite> with key <cat>
layer <4/multipolygons> table <roads_4> in database
</home/mneteler/grassdata/latlong_wgs84/osm/sqlite/sqlite.db> through
driver <sqlite> with key <cat>
layer <5/other_relations> table <roads_5> in database
</home/mneteler/grassdata/latlong_wgs84/osm/sqlite/sqlite.db> through
driver <sqlite> with key <cat>
Problem:
Only vector queries in the GUI fail with a JSON error. It will be some
"escape" issue.
I have opened a ticket for that: https://trac.osgeo.org/grass/ticket/3409
BTW, with GDAL 2.2.1, using OGR_INTERLEAVED_READING=YES does not work for me
(empty vector).
I got here:
gdal-config --version
2.1.3
v.in.ogr
...
Check if OGR layer <points> contains polygons...
ERROR 1: Too many features have accumulated in lines layer. Use
OGR_INTERLEAVED_READING=YES mode
Warning 1: organizePolygons() received an unexpected geometry.
Either a polygon with interior rings, or a polygon with less than 4 points,
or a non-Polygon geometry. Return arguments as a collection.
100%
Check if OGR layer <lines> contains polygons...
100%
Check if OGR layer <multilinestrings> contains polygons...
100%
Check if OGR layer <multipolygons> contains polygons...
100%
Check if OGR layer <other_relations> contains polygons...
100%
...
BTW, with GDAL 2.2.1, using OGR_INTERLEAVED_READING=YES does not work for
me (empty vector).
Caution: you cannot just specify OGR_INTERLEAVED_READING=YES and loop over layers as usual. See the “Interleaved reading” section of http://gdal.org/drv_osm.html
If you want to convert centroids to points and boundaries to lines, use type=point,line.
If you want to import only points and lines from the OSM PBF file, use layer=points,lines (check layer names first with v.in.ogr -l).
Markus M
…
Number of nodes: 177060
Number of primitives: 743673
Number of points: 550356
Number of lines: 193317
Number of boundaries: 0
Number of centroids: 0
Number of areas: 0
Number of isles: 0
Nice!
GRASS 7.2.2svn (latlong_wgs84):~ > v.db.connect -p roads
Vector map is connected by:
layer <1/points> table <roads_1> in database
</home/mneteler/grassdata/latlong_wgs84/osm/sqlite/sqlite.db> through
driver with key
layer <2/lines> table <roads_2> in database
</home/mneteler/grassdata/latlong_wgs84/osm/sqlite/sqlite.db> through
driver with key
layer <3/multilinestrings> table <roads_3> in database
</home/mneteler/grassdata/latlong_wgs84/osm/sqlite/sqlite.db> through
driver with key
layer <4/multipolygons> table <roads_4> in database
</home/mneteler/grassdata/latlong_wgs84/osm/sqlite/sqlite.db> through
driver with key
layer <5/other_relations> table <roads_5> in database
</home/mneteler/grassdata/latlong_wgs84/osm/sqlite/sqlite.db> through
driver with key
Problem:
Only vector queries in the GUI fail with a JSON error. It will be some
“escape” issue.
I have opened a ticket for that: https://trac.osgeo.org/grass/ticket/3409
BTW, with GDAL 2.2.1, using OGR_INTERLEAVED_READING=YES does not work for me
(empty vector).
I got here:
gdal-config --version
2.1.3
v.in.ogr
…
Check if OGR layer contains polygons…
ERROR 1: Too many features have accumulated in lines layer. Use
OGR_INTERLEAVED_READING=YES mode
Warning 1: organizePolygons() received an unexpected geometry.
Either a polygon with interior rings, or a polygon with less than 4 points,
or a non-Polygon geometry. Return arguments as a collection.
100%
Check if OGR layer contains polygons…
100%
Check if OGR layer contains polygons…
100%
Check if OGR layer contains polygons…
100%
Check if OGR layer <other_relations> contains polygons…
100%
…
BTW, with GDAL 2.2.1, using OGR_INTERLEAVED_READING=YES does not work for
me (empty vector).
Caution: you cannot just specify OGR_INTERLEAVED_READING=YES and loop over layers as usual. See the “Interleaved reading” section of http://gdal.org/drv_osm.html
I see, thanks for the hint. That means v.in.ogr must be changed to use the special reading mode, or OGR_INTERLEAVED_READING must be unset in v.in.ogr, together with a warning.
I see, thanks for the hint. That means v.in.ogr must be changed to use the
special reading mode, or OGR_INTERLEAVED_READING must be unset in v.in.ogr,
together with a warning.
If you unset OGR_INTERLEAVED_READING, the classical read pattern (loop over layers, and for each layer, loop over its features) will work only for small osm/pbf files where the number of features in each layer isn’t larger than 100K.
For larger files, OGR_INTERLEAVED_READING=YES and the special reading pattern described in drv_osm.html is needed.
I see, thanks for the hint. That means v.in.ogr must be changed to use the
special reading mode, or OGR_INTERLEAVED_READING must be unset in v.in.ogr,
together with a warning.
If you unset OGR_INTERLEAVED_READING, the classical read pattern (loop over layers, and for each layer, loop over its features) will work only for small osm/pbf files where the number of features in each layer isn’t larger than 100K.
For larger files, OGR_INTERLEAVED_READING=YES and the special reading pattern described in drv_osm.html is needed.
OK, I have disabled OSM import for trunk and relbr72 (r71543,4), because it results in empty vectors or incomplete imports.
I am not sure if v.in.ogr should be modified to use this special reading mode for one single OGR supported format. The GDAL/OGR API is very convenient because applications do not need to bother about particular formats. If we start implementing code for particular formats in v.in.ogr, we might open a Pandora’s box…
I am not sure if v.in.ogr should be modified to use this special reading mode for one single OGR supported format. The GDAL/OGR API is very convenient because applications do not need to bother about particular formats. If we start implementing code for particular formats in v.in.ogr, we might open a Pandora’s box…
Interleaved reading for OSM input has been added to v.in.ogr in trunk r71464.
OSM import remains challenging. For v.in.ogr you need define a custom OSM_CONFIG_FILE because the default osmconf.ini file will add all tags not defined with attributes into “other_tags”, potentially causing buffer overflow. OSM polygons are overlapping which is correct because e.g. administrative boundaries for multiple administrative levels are imported.
On Mon, Sep 4, 2017 at 8:32 PM, Markus Metz
<markus.metz.giswork@gmail.com> wrote:
On Thu, Aug 31, 2017 at 7:21 PM, Markus Metz <markus.metz.giswork@gmail.com>
I am not sure if v.in.ogr should be modified to use this special reading
mode for one single OGR supported format. The GDAL/OGR API is very
convenient because applications do not need to bother about particular
formats. If we start implementing code for particular formats in v.in.ogr,
we might open a Pandora's box...
Interleaved reading for OSM input has been added to v.in.ogr in trunk
r71464.
Thanks for that!
OSM import remains challenging. For v.in.ogr you need define a custom
OSM_CONFIG_FILE because the default osmconf.ini file will add all tags not
defined with attributes into "other_tags", potentially causing buffer
overflow. OSM polygons are overlapping which is correct because e.g.
administrative boundaries for multiple administrative levels are imported.
Perhaps we can cook up some examples in the GRASS Wiki for common use
cases (or add to the manual page of v.in.ogr)?
I am not sure if v.in.ogr should be modified to use this special reading
mode for one single OGR supported format. The GDAL/OGR API is very
convenient because applications do not need to bother about particular
formats. If we start implementing code for particular formats in v.in.ogr,
we might open a Pandora’s box…
Interleaved reading for OSM input has been added to v.in.ogr in trunk
r71464.
Thanks for that!
v.in.ogr has been further updated in trunk r71532 to use new features available with GDAL 2.2.
BTW, when translating OSM data with ogr2ogr, the results might differ between GDAL 2.1 and GDAL 2.2. Similarly, when importing OSM data with v.in.ogr, results differ between GDAL 2.1 and GDAL 2.2, but are equivalent to the corresponding ogr2ogr version.
OSM import remains challenging. For v.in.ogr you need define a custom
OSM_CONFIG_FILE because the default osmconf.ini file will add all tags not
defined with attributes into “other_tags”, potentially causing buffer
overflow. OSM polygons are overlapping which is correct because e.g.
administrative boundaries for multiple administrative levels are imported.
Perhaps we can cook up some examples in the GRASS Wiki for common use
cases (or add to the manual page of v.in.ogr)?