[GRASS-user] creating high-resolution DMT

GRASS GRASS user
1

Hi,

recently I was playing with DMT products available in Czech Republic, basically:

* DMR4G regular grid of points (spacing 5m)
* DMR5G irregular point network (average spacing ~3m)

When creating raster-based DMT I tested scenarios bellow:

1) create raster with resolution 5m from DMR4G (no interpolation, just
g.region + r.in.xyz/r.in.lidar)

2) create raster with high resolution (less than 3m) with v.surf.rst
(default parameters defined)

Unfortunately v.surf.rst is quite slow (tile with 5e5 points and
region 2000x2500 cells took on my pc about 3hours). I enjoyed very
much parallelization support in GRASS 7.3. Using 8 cores I am able to
process one tile in 20min. It's great. Probably also some tunning
parameters could help(?)

I was thinking about other scenarios as:

3) import points using r.in.xyz/r.in.lidar on desired resolution and
fill holes with r.fillnuls (but it will be also slow)

4) use v.surf.bspline

What is your experience and how you would process such data?

Thanks for your feedback in advance! Martin

--
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa

2

Hei Martin,

Same experience here.
For larger datasets from LiDAR I rather use IDW for interpolation.

The "per hole filling" in r.fillnulls is not utilizing multiple cores and if you have a lot of holes that can slow down the whole process significantly.
Not sure if a mask can help to avoid filling no data areas around your data.

For r.fillnulls there is an enhancement ticket (with a patch) for speedup of the patching of the maps at the end...
https://trac.osgeo.org/grass/ticket/1938
However, it does not tackle parallel filling of holes...

Cheers
Stefan

3

On 21/03/17 10:12, Blumentrath, Stefan wrote:

Hei Martin,

Same experience here. For larger datasets from LiDAR I rather use IDW
for interpolation.

I just realized that I have never committed "r.fill.gaps"
to the add-ins repo.

I have just done that now:

http://grasswiki.osgeo.org/wiki/AddOns/GRASS_6#r.fill.gaps

Its purpose is filling small gaps in otherwise dense
data using IDW with a pre-computed weights matrix.
It works on rasterized data, so you simply use v.to.rast
on your vector points first. Make sure to set the
cell size small enough so that you don't get many
vector points falling into the same cell. The result
will be an oversampled raster with a lot of small
"no data" cells. That's exactly the intended input for
r.fill.gaps!

This is not multi-core code, (parallelizing
interpolation algorithms is hard, because you need to
segment the data and then you need to deal with the
seams between the segments), but it is very, very fast,
as long as you keep the IDW radius small.

The code is optimized to death, which makes it very
hard to read. Another drawback is that it was written
for GRASS 6 (but converting it to GRASS 7 should not
be hard, since it uses only the basic raster API).

r.fill.gaps is not useful for filling large gaps, both
in terms of performance (for large IDW radii) and
interpolation quality (it's IDW -- enough said).

Best,

Ben

The "per hole filling" in r.fillnulls is not utilizing multiple cores
and if you have a lot of holes that can slow down the whole process
significantly. Not sure if a mask can help to avoid filling no data
areas around your data.

For r.fillnulls there is an enhancement ticket (with a patch) for
speedup of the patching of the maps at the end...
https://trac.osgeo.org/grass/ticket/1938 However, it does not tackle
parallel filling of holes...

Cheers Stefan

_______________________________________________ grass-user mailing
list grass-user@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-user

--
Dr. Benjamin Ducke
{*} Geospatial Consultant
{*} GIS Developer

Spatial technology for the masses, not the classes:
experience free and open source GIS at http://gvsigce.org

4

On Tue, Mar 21, 2017 at 12:29 PM, Benjamin Ducke <benducke@fastmail.fm> wrote:
...

I just realized that I have never committed "r.fill.gaps"
to the add-ins repo.

I have just done that now:
http://grasswiki.osgeo.org/wiki/AddOns/GRASS_6#r.fill.gaps

cool!

...

The code is optimized to death, which makes it very
hard to read. Another drawback is that it was written
for GRASS 6 (but converting it to GRASS 7 should not
be hard, since it uses only the basic raster API).

FWIW New names can be looked up here:
https://trac.osgeo.org/grass/wiki/Grass7/RasterLib/ListOfFunctions

Best
Markus

5

Back to filling holes in lidar data, I also think that the "fill in the
gaps" approach is quite promising, so I ported the r.fill.gaps module to G7:

https://grass.osgeo.org/grass72/manuals/addons/r.fill.gaps.html

I didn't review the code, but the documentation is very nice and detailed.
I did some mostly formatting updates for the current submitting guidelines
and added complete example with lidar data.

On Tue, Mar 21, 2017 at 7:29 AM, Benjamin Ducke <benducke@fastmail.fm>
wrote:

I just realized that I have never committed "r.fill.gaps"
to the add-ins repo.

Really nice module, please, find more of these in your shelf :slight_smile:

Thanks!
Vaclav

I have just done that now:

http://grasswiki.osgeo.org/wiki/AddOns/GRASS_6#r.fill.gaps

Its purpose is filling small gaps in otherwise dense
data using IDW with a pre-computed weights matrix.
It works on rasterized data, so you simply use v.to.rast
on your vector points first. Make sure to set the
cell size small enough so that you don't get many
vector points falling into the same cell. The result
will be an oversampled raster with a lot of small
"no data" cells. That's exactly the intended input for
r.fill.gaps!

This is not multi-core code, (parallelizing
interpolation algorithms is hard, because you need to
segment the data and then you need to deal with the
seams between the segments), but it is very, very fast,
as long as you keep the IDW radius small.

The code is optimized to death, which makes it very
hard to read. Another drawback is that it was written
for GRASS 6 (but converting it to GRASS 7 should not
be hard, since it uses only the basic raster API).

r.fill.gaps is not useful for filling large gaps, both
in terms of performance (for large IDW radii) and
interpolation quality (it's IDW -- enough said).

6

On Tue, Mar 21, 2017 at 10:01 AM, Markus Neteler <neteler@osgeo.org> wrote:

Another drawback is that it was written
for GRASS 6 (but converting it to GRASS 7 should not
be hard, since it uses only the basic raster API).

FWIW New names can be looked up here:
https://trac.osgeo.org/grass/wiki/Grass7/RasterLib/ListOfFunctions

For those interested in what was required for the port from 6 to 7:

https://trac.osgeo.org/grass/changeset/70880

7

On 15/04/17 05:31, Vaclav Petras wrote:

Back to filling holes in lidar data, I also think that the "fill in the
gaps" approach is quite promising, so I ported the r.fill.gaps module to G7:

Great! I have read the diff and that was very
helpful for me. I can see that the old and
new raster APIs have only minimal differences.
Porting seems very straight-forward.

I hope that r.fill.gaps will be useful for others.
I did not have LiDAR in mind when I wrote it,
but now it seems so obvious that LiDAR processing
is an important domain of application for the module.

Cheers!

Ben

https://grass.osgeo.org/grass72/manuals/addons/r.fill.gaps.html

I didn't review the code, but the documentation is very nice and
detailed. I did some mostly formatting updates for the current
submitting guidelines and added complete example with lidar data.

On Tue, Mar 21, 2017 at 7:29 AM, Benjamin Ducke <benducke@fastmail.fm
<mailto:benducke@fastmail.fm>> wrote:

    I just realized that I have never committed "r.fill.gaps"
    to the add-ins repo.

Really nice module, please, find more of these in your shelf :slight_smile:

Thanks!
Vaclav

    I have just done that now:

    http://grasswiki.osgeo.org/wiki/AddOns/GRASS_6#r.fill.gaps
    <http://grasswiki.osgeo.org/wiki/AddOns/GRASS_6#r.fill.gaps&gt;

    Its purpose is filling small gaps in otherwise dense
    data using IDW with a pre-computed weights matrix.
    It works on rasterized data, so you simply use v.to.rast
    on your vector points first. Make sure to set the
    cell size small enough so that you don't get many
    vector points falling into the same cell. The result
    will be an oversampled raster with a lot of small
    "no data" cells. That's exactly the intended input for
    r.fill.gaps!

    This is not multi-core code, (parallelizing
    interpolation algorithms is hard, because you need to
    segment the data and then you need to deal with the
    seams between the segments), but it is very, very fast,
    as long as you keep the IDW radius small.

    The code is optimized to death, which makes it very
    hard to read. Another drawback is that it was written
    for GRASS 6 (but converting it to GRASS 7 should not
    be hard, since it uses only the basic raster API).

    r.fill.gaps is not useful for filling large gaps, both
    in terms of performance (for large IDW radii) and
    interpolation quality (it's IDW -- enough said).

--
Dr. Benjamin Ducke
{*} Geospatial Consultant
{*} GIS Developer

Spatial technology for the masses, not the classes:
experience free and open source GIS at http://gvsigce.org