Hi all,
r.in.lidar in trunk (r66601) can now read multiple LAS files specified in a text file (similarly to what e.g. r.colors supports for raster maps). It is nothing fancy, no indexing or something, but you can bin point from several tiles without any edge effects. It is not particularly fast either, but it is not worse than the original code.
Here is an example with custom resolution and extent derived from the data:
r.in.lidar file=list.txt output=raster method=mean -e res=30 zrange=-1,200
The file list.txt contains something like:
/home/johnuser/data/tile_55.las
/home/johnuser/data/tile_56.las
/home/johnuser/data/tile_57.las
A major refactoring of the code (r66593-r66596) was necessary for smooth implementation. I did my best to test different combination of parameters with comparison to original results, but I was not able to create an automated test partially for the lack of standardized input data. More testing appreciated.
The options and tabs were reorganized to accommodate new inputs, but the documentation is missing.
Let me know what you think.
Vaclav
https://grass.osgeo.org/grass71/manuals/r.in.lidar.html
https://trac.osgeo.org/grass/changeset/66601
https://trac.osgeo.org/grass/changeset/66593
https://trac.osgeo.org/grass/changeset/66594
https://trac.osgeo.org/grass/changeset/66595
https://trac.osgeo.org/grass/changeset/66596
Vaclav,
I cannot wait to try this with multiple files at the 4.2 point las 1.2 limit! I can live with slow, just concerned about the memory demands working with larger areas.
Thank you very much!
Doug
···
On Mon, Oct 26, 2015 at 11:28 AM, Vaclav Petras <wenzeslaus@gmail.com> wrote:
Hi all,
r.in.lidar in trunk (r66601) can now read multiple LAS files specified in a text file (similarly to what e.g. r.colors supports for raster maps). It is nothing fancy, no indexing or something, but you can bin point from several tiles without any edge effects. It is not particularly fast either, but it is not worse than the original code.
Here is an example with custom resolution and extent derived from the data:
r.in.lidar file=list.txt output=raster method=mean -e res=30 zrange=-1,200
The file list.txt contains something like:
/home/johnuser/data/tile_55.las
/home/johnuser/data/tile_56.las
/home/johnuser/data/tile_57.las
A major refactoring of the code (r66593-r66596) was necessary for smooth implementation. I did my best to test different combination of parameters with comparison to original results, but I was not able to create an automated test partially for the lack of standardized input data. More testing appreciated.
The options and tabs were reorganized to accommodate new inputs, but the documentation is missing.
Let me know what you think.
Vaclav
https://grass.osgeo.org/grass71/manuals/r.in.lidar.html
https://trac.osgeo.org/grass/changeset/66601
https://trac.osgeo.org/grass/changeset/66593
https://trac.osgeo.org/grass/changeset/66594
https://trac.osgeo.org/grass/changeset/66595
https://trac.osgeo.org/grass/changeset/66596
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
–
Doug Newcomb
USFWS
Raleigh, NC
919-856-4520 ext. 14 doug_newcomb@fws.gov
The opinions I express are my own and are not representative of the official policy of the U.S.Fish and Wildlife Service or Dept. of the Interior. Life is too short for undocumented, proprietary data formats. As a federal employee, my email may be subject to FOIA request.
On Mon, Oct 26, 2015 at 1:20 PM, Newcomb, Doug <doug_newcomb@fws.gov> wrote:
multiple files at the 4.2 point las 1.2 limit! I can live with slow, just
concerned about the memory demands working with larger areas.
The processing goes by row chunks. By default everything is in the memory.
You can say that only half of rows will be in the memory at a time by
percent=50 (you have to guess how much memory it will take). It will be
(much) slower because it will iterate over the files (twice in this case).
I was not able to do any tests with large amounts of points but the only
problem will be capacity of some counters for percentage and report. The
actual processing takes one point at a time, so there shouldn't be any
limit. This is by the way still the same as before, I haven't changed
anything in the way how this is handled. Let me know if this doesn't
address your concerns.
Vaclav,
I have successfully processed a county with 966 laz files with well over 4.2 billion points, using a base elevation raster with 2.4 billion pixels ( Hyde county, NC 5 ft resolution DEM , about half water). It is a bit slower, but it does solve several of my issues. Well done!
Will this be backported to 7.0 ? 
Doug
···
On Mon, Oct 26, 2015 at 3:00 PM, Vaclav Petras <wenzeslaus@gmail.com> wrote:
–
On Mon, Oct 26, 2015 at 1:20 PM, Newcomb, Doug <doug_newcomb@fws.gov> wrote:
multiple files at the 4.2 point las 1.2 limit! I can live with slow, just concerned about the memory demands working with larger areas.
The processing goes by row chunks. By default everything is in the memory. You can say that only half of rows will be in the memory at a time by percent=50 (you have to guess how much memory it will take). It will be (much) slower because it will iterate over the files (twice in this case). I was not able to do any tests with large amounts of points but the only problem will be capacity of some counters for percentage and report. The actual processing takes one point at a time, so there shouldn’t be any limit. This is by the way still the same as before, I haven’t changed anything in the way how this is handled. Let me know if this doesn’t address your concerns.
Doug Newcomb
USFWS
Raleigh, NC
919-856-4520 ext. 14 doug_newcomb@fws.gov
The opinions I express are my own and are not representative of the official policy of the U.S.Fish and Wildlife Service or Dept. of the Interior. Life is too short for undocumented, proprietary data formats. As a federal employee, my email may be subject to FOIA request.
On Fri, Oct 30, 2015 at 8:03 AM, Newcomb, Doug <doug_newcomb@fws.gov> wrote:
I have successfully processed a county with 966 laz files with well over
4.2 billion points, using a base elevation raster with 2.4 billion pixels (
Hyde county, NC 5 ft resolution DEM , about half water). It is a bit
slower, but it does solve several of my issues. Well done!
Thanks for testing it!
Will this be backported to 7.0 ?
Sorry. No plans to do that. It is a feature and significant change of the
code. This is exactly something which shouldn't be backported unless we are
absolutely sure that it works. We should however make sure that we can
release 7.1 soon enough.