On Mon, Apr 14, 2008 at 1:33 AM, Hamish <hamish_b@yahoo.com> wrote:
Mark wrote:
The points are about 5 million per tile, and there are 23 tiles (about
115 million total). I’m not sure if this is considered a large amount
of mass input points.
Yes, it’s a lot. The current vector engine allocates a small amount of
memory per feature for topology. With the DBF as database and default
v.in.ascii settings I’ve only ever managed to import about 3 million
points before running out of memory. I am interested to hear that you
could load 5 million into a SQLite DB, maybe that is more efficient.
The SQlite setup was so easy (thanks to the book), and then able to start working with the tables was a breeze (literally as easy as db.connect driver=sqlite database="$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite.db.) I’m not a SQL expert, but learned enough to do some sweet tasks. SQlite was rather impressive in working with 5.2 million records. Through v.in.ascii, it did load all the data (4 columns) into the sqlite database without problem (about 7 minutes on a 64bit 2.0GHz 2GB ram system - no table and no topology)
With the v.in.ascii -t and -b flags you should be able to load 25million+
points into a single vector map, but there is a limited number of modules
that will be able to use a vector map without topology. (importantly
v.surf.rst still works)
Ayone know if the v.outlier, v.lidar.edgedetect, v. lidar.grow and v.lidar.correction need topology?
If you are willing to abandon your “code” data column, you can strip off
that column and import the rest as a 3D vector (-z z=), in which case no
table is created (or just ignore it with -t -z). If you are just
interested in the 3D coordinate, then a DB table is unneeded overhead and
is best skipped.
This approach turned out to be the best. You (Hamish) were right that the code was an intensity or strength of return signal. I made a nice “intensity” grayscale map, that was great. Since I dont need the intensity, or know how to use it to help separate the DEM from DSM features, the 3D point worked perfect. Thanks!
Question: what do you want to do with the data? Simply create a raster
DEM or do more fancy cleaning with v.lidar.*? If just creating a raster
DEM you might skip v.in.ascii altogether and use the r.in.xyz module.
I will definitely look more closely at the r.in.xyz module. I had used it for some binning and statistical analysis of the point distribution of the lidar. it is pretty tight data.
I want/need to do fancy cleaning with the v.lidar.* tools. However, not having pulse return counts to work with like the example in the book, I am not sure how well it will work, because only having an intensity value, I’m not sure how that relates to first and last returns. What I would like to do with the data is create a bare earth DEM, and a DSM of the elevated surface features. Then perhaps add elevated surface features like solid objects (e.g. buildings, etc) that would block terrain for shallow overland flow or r.lake simulations for flooding.
Since the application is for watershed analysis (feature building, and ultimately running hydrology models in GRASS), so my goal is to create the most realistic hydrologic surface model to properly route the water. In theory, it seems to me that water should not route through solid features like buildings (extrude as walls?), but should be able to (and often does) route through shrubby or forested areas. In these areas, where there is canopy, it seems that those canopy elevations should be replaced with bare earth elevations, but allow water to flow through these areas. This is the first time I have worked with raw LiDAR data to create features or do watershed analysis, so this is a fun learning experience. Typically, I had worked with contour data in the past.
Mark