#2651: Increase maximum number of vector points available
-------------------------------+--------------------------------------------
Reporter: dnewcomb | Owner: grass-dev@…
Type: enhancement | Status: new
Priority: normal | Milestone: 7.1.0
Component: Vector | Version: unspecified
Keywords: vector point data | Platform: Unspecified
Cpu: Unspecified |
-------------------------------+--------------------------------------------
The current number of points in a imported into a vector data layer
appears to be limited to a 32 bit integer ( 2 billion) . You are allowed
to import more, but the number of points are indicated with a negative
number Not sure if the data set is still valid at that point.
The primary use case for this is lidar data. The new USGS standard for
landscape LiDAR data collection is QL2 , http://pubs.usgs.gov/tm/11b4/ ,
roughly 2 points/meter density. The lidar ground points at this density
can exceed the 32 bit integer limit in a 1000 km^2 area (31km x 31km). I
already have counties in NC, USA with ground point data that exceed this
threshold.
Newer LiDAR technologies ( Single Photon, Geiger Mode) with collection
densities of 20 points/meter are already available, but r.in.lidar may be
more suitable for these data sets in the short term.
#2651: Increase maximum number of vector points available
--------------------------------------+-------------------------------------
Reporter: dnewcomb | Owner: grass-dev@…
Type: enhancement | Status: new
Priority: normal | Milestone: 7.1.0
Component: Vector | Version: svn-trunk
Keywords: lidar, vector point data | Platform: Unspecified
Cpu: Unspecified |
--------------------------------------+-------------------------------------
Changes (by neteler):
* keywords: vector point data => lidar, vector point data
* version: unspecified => svn-trunk
Comment:
Replying to [ticket:2651 dnewcomb]:
> The current number of points in a imported into a vector data layer
appears to be limited to a 32 bit integer ( 2 billion).
Isn't this the limit in case of topology being enabled? Did you try to
disable topology during import?
#2651: Increase maximum number of vector points available
--------------------------------------+-------------------------------------
Reporter: dnewcomb | Owner: grass-dev@…
Type: enhancement | Status: new
Priority: normal | Milestone: 7.1.0
Component: Vector | Version: svn-trunk
Keywords: lidar, vector point data | Platform: Unspecified
Cpu: Unspecified |
--------------------------------------+-------------------------------------
Comment(by dnewcomb):
Topology was disabled. Attribute table was not built. Looking back, I am
running an svn snapshot from 1-10-2015, was this addressed before the
final release?
#2651: Increase maximum number of vector points available
--------------------------------------+-------------------------------------
Reporter: dnewcomb | Owner: grass-dev@…
Type: enhancement | Status: new
Priority: normal | Milestone: 7.1.0
Component: Vector | Version: svn-trunk
Keywords: lidar, vector point data | Platform: Unspecified
Cpu: Unspecified |
--------------------------------------+-------------------------------------
Comment(by dnewcomb):
Installed 4-4-15 weekly svn snapshot and started import of las file with
roughly 2.4 billion points. v.in.lidar is reporting scanning -1.87
billion points on import.
#2651: Increase maximum number of vector points available
--------------------------------------+-------------------------------------
Reporter: dnewcomb | Owner: grass-dev@…
Type: enhancement | Status: new
Priority: normal | Milestone: 7.1.0
Component: Vector | Version: svn-trunk
Keywords: lidar, vector point data | Platform: Unspecified
Cpu: Unspecified |
--------------------------------------+-------------------------------------
Comment(by marisn):
Could you, please, copy / paste exact message text?
It could be as simple as wrong formatting character used while printing
message or overflowing counter used just to print progress.
#2651: Increase maximum number of vector points available
--------------------------------------+-------------------------------------
Reporter: dnewcomb | Owner: grass-dev@…
Type: enhancement | Status: new
Priority: normal | Milestone: 7.1.0
Component: Vector | Version: svn-trunk
Keywords: lidar, vector point data | Platform: Unspecified
Cpu: Unspecified |
--------------------------------------+-------------------------------------
Comment(by dnewcomb):
Replying to [comment:6 marisn]:
> Could you, please, copy / paste exact message text?
> It could be as simple as wrong formatting character used while printing
message or overflowing counter used just to print progress.
When I ran v.in.lidar on the previous version of GRASS 7 that I was using,
I ran v.info on the resulting vector data set and go the same number for
the number of points .
Here is the info on the lidar file:
newcomb@IFW4FO-RAL-SGIS:/media/Seagate Expansion Drive/BladenCoNC$
~/lastools/lastools/bin/lasinfo bladen_ground0.laz
reporting all LAS header entries:
file signature: 'LASF'
file source ID: 0
global_encoding: 1
project ID GUID data 1-4: 00000000-0000-0000-0000-000000000000
version major.minor: 1.3
system identifier: 'LAStools (c) by Martin Isenburg'
generating software: 'lasmerge (version 131210)'
file creation day/year: 252/2014
header size: 235
offset to point data: 345
number var. length records: 1
point data format: 3
point data record length: 34
number of point records: 2415002259
number of points by return: 1503313306 704621650 204714902 2216754
135647
scale factor x y z: 0.01 0.01 0.01
offset x y z: -0 -0 -0
min x y z: 2030000.00 225000.00 3.92
max x y z: 2244999.99 404999.99 174.24
#2651: Increase maximum number of vector points available
--------------------------------------+-------------------------------------
Reporter: dnewcomb | Owner: grass-dev@…
Type: enhancement | Status: new
Priority: normal | Milestone: 7.1.0
Component: Vector | Version: svn-trunk
Keywords: lidar, vector point data | Platform: Unspecified
Cpu: Unspecified |
--------------------------------------+-------------------------------------
Comment(by dnewcomb):
Replying to [comment:7 dnewcomb]:
Looking back, this seems to be a duplication of this ticket, which I
forgotten I had submitted http://trac.osgeo.org/grass/ticket/2472
> Replying to [comment:6 marisn]:
> > Could you, please, copy / paste exact message text?
> > It could be as simple as wrong formatting character used while
printing message or overflowing counter used just to print progress.
>
> v.in.lidar -t -o -b input=/media/Seagate Expansion
Drive/BladenCoNC/bladen_ground0.laz output=bladen_ground_pts
> Over-riding projection check
> Scanning -1879965037 points...
>
> When I ran v.in.lidar on the previous version of GRASS 7 that I was
using, I ran v.info on the resulting vector data set and go the same
number for the number of points .
>
> Here is the info on the lidar file:
>
> newcomb@IFW4FO-RAL-SGIS:/media/Seagate Expansion Drive/BladenCoNC$
~/lastools/lastools/bin/lasinfo bladen_ground0.laz
> reporting all LAS header entries:
> file signature: 'LASF'
> file source ID: 0
> global_encoding: 1
> project ID GUID data 1-4: 00000000-0000-0000-0000-000000000000
> version major.minor: 1.3
> system identifier: 'LAStools (c) by Martin Isenburg'
> generating software: 'lasmerge (version 131210)'
> file creation day/year: 252/2014
> header size: 235
> offset to point data: 345
> number var. length records: 1
> point data format: 3
> point data record length: 34
> number of point records: 2415002259
> number of points by return: 1503313306 704621650 204714902 2216754
135647
> scale factor x y z: 0.01 0.01 0.01
> offset x y z: -0 -0 -0
> min x y z: 2030000.00 225000.00 3.92
> max x y z: 2244999.99 404999.99 174.