[GRASS-dev] [GRASS GIS] #1088: r.fillnulls: support other interpolation methods

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------
Support other interpolation methods in r.fillnulls, like bilinear and
bicubic. This can be done from v.surf.bspline. Patch attached for dev6
branch.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1088&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by kyngchaos):

...I should make sure it works before attaching the patch...

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1088#comment:1&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: Unspecified
      Cpu: Unspecified |
-------------------------+--------------------------------------------------

Comment(by kyngchaos):

Now the patch works.

I forgot to mention, the idea is that the rst method may not work well on
large areas (blocks from segmentation don't align well). bilinear/bicubic
seems to process smoother.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1088#comment:2&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------
Changes (by hamish):

  * platform: Unspecified => All
  * cpu: Unspecified => All

Comment:

thanks, this has been on the todo list for a long time (I 1/2 thought
there was already a ticket for it, but guess not).

here's a question (maybe for Markus Metz): what method should be the
default in grass 7?

porting this patch to python should be trivial.

cheers,
Hamish

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1088#comment:3&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------

Comment(by kyngchaos):

One thing that would be nice in v.surf.bspline is for it to use the MASK,
like v.surf.rst does, otherwise there can be a lot of warnings about no
data in a subregion when I want to mask off areas that I want to stay
null. (there's a note in the patch about the MASK not applying, but I set
it anyways for the future possibility)

I was going to add a feature request for this, but I'm burnt out today
from testing the processing of the patch.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1088#comment:4&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------

Comment(by hamish):

MASKs used in GRASS are only used when _reading_ existing raster maps.
They do not mask writing. If v.surf.rst respects MASKs at all it would be
somewhat surprising news to me and should be well documented.

As the output raster map is only written for the current region, ignoring
points outside of this box (and maybe a little buffer around it) can save
some needless work, and so v.surf.rst does do that.

Hamish

ps- note to self:
{{{
- g.rename > /dev/null
+ g.rename --quiet
}}}

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1088#comment:5&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------

Comment(by kyngchaos):

Sorry, I was being vague, v.surf.rst has an option to set a mask for
writing (though I didn't realize that MASKs only apply to reading, thanks,
though it makes sense).

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1088#comment:6&gt;
GRASS GIS <http://grass.osgeo.org>

GRASS GIS wrote:

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: Unspecified
Cpu: Unspecified |
-------------------------+--------------------------------------------------
Support other interpolation methods in r.fillnulls, like bilinear and
bicubic. This can be done from v.surf.bspline. Patch attached for dev6
branch.

Just a quick note: it's already implemented in r.resamp.bspline in
7.0. r.resamp.bspline interpolates a raster to the current resolution,
filling NULL cells on the fly, or optionally only interpolates NULL
cells. Available methods are bilinear and bicubic, tested with SRTM
data, filling large gaps in the European Alps.

Proper report in trac once I'm back in office.

Markus M

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------

Comment(by mmetz):

In grass 7, it's already implemented in r.resamp.bspline. r.resamp.bspline
interpolates a raster to the current resolution, filling NULL cells on the
fly, or optionally only interpolates NULL
cells. Available methods are bilinear and bicubic, tested with SRTM data,
filling large gaps in the European Alps.

For grass 6.x, I would use a different approach to r.fillnulls to follow
the design of v.surf.bspline, because only interpolating NULL cells is
already possible with v.surf.bspline:

Create a new raster where NULL cells in the original surface raster are
set to something else than NULL, others to NULL. Convert both raster maps
to vector points. Use the vector points representing NULL cells as sparse
points input for v.surf.bspline. Recommended settings for v.surf.bspline
in this case are sie = 2 * ewres, sin = 2 * nsres, lambda = 0.005 (best in
my tests, should be somewhere between 0.001 and 0.01). The output raster
holds the interpolated NULL cells and can be patched with the original
surface.

Although all lidar tools work within the current region, none respects a
MASK. I'm not sure how this could be implemented properly and efficiently,
currently there will be IMHO harmless warnings about no points in the
current subregion.

Markus M

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1088#comment:7&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------

Comment(by kyngchaos):

Markus, see my patch. It's very similar to what you describe. A couple
notes/questions:

I didn't completely understand the description for the sparse option, but
it didn't look like it was needed here. Now it looks to me like it's
essentially a mask option, like the maskmap option in v.surf.rst. For
now, the mask is implied when the results are patched into the original
raster. I guess a mask option (if sparse is such) only makes sense if it
reduces processing time by ignoring the masked areas.

The description says 1 * resolution was good for regularly spaced points,
which is what we get from a raster. Why 2* in your method?

For the lambda, the docs are unclear what orders of numbers have effects,
beyond "the larger the number the more smoothing applied". The default is
one. You say 0.001-0.01 work well. I figured 1 was something like 1*, or
no smoothing. Can you clarify this option?

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1088#comment:8&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------

Comment(by mmetz):

Replying to [comment:8 kyngchaos]:
> A couple notes/questions:
>
> I didn't completely understand the description for the sparse option,
but it didn't look like it was needed here. Now it looks to me like it's
essentially a mask option, like the maskmap option in v.surf.rst. For
now, the mask is implied when the results are patched into the original
raster. I guess a mask option (if sparse is such) only makes sense if it
reduces processing time by ignoring the masked areas.

Right. Unfortunately, using sparse points as input for v.surf.bspline does
not reduce processing time substantially, although it could be implemented
similar to what I did for r.resamp.bspline. The sparse points act somewhat
similar to a mask in the way that only these sparse points are
interpolated. The matrix calculations are done anyway, a waste of time if
no sparse points are within the current subregion. Therefore I think your
approach is much faster for just a few NULL cells in a large surface
raster.
>
> The description says 1 * resolution was good for regularly spaced
points, which is what we get from a raster. Why 2* in your method?

This is a compromise between speed, smoothing, and interpolating both
small and large gaps. The spline step is similar to tension in RST, larger
spline step values mean that points influence each other over larger
distances. 1 * resolution seems to be the best for very small gaps, but
for larger steps, I would use larger spline steps. I did a bit of testing,
punching holes into a DEM and interpolating these holes. Spline steps of
1.5 - 2 * resolution produced best results, i.e. interpolated values were
closest to real values.

I have to update the manual for r.resamp.bspline because it uses 1.5 *
resolution as default spline step. For a gap-free surface, I think 1 *
resolution of the input raster is best
>
> For the lambda, the docs are unclear what orders of numbers have
effects, beyond "the larger the number the more smoothing applied". The
default is one. You say 0.001-0.01 work well. I figured 1 was something
like 1*, or no smoothing. Can you clarify this option?

Umm, that's trial and error what I did there. Lambda = 1 did usually not
provide very nice results, too much smoothing. Lambda and spline step
influence each other, I tried to find reasonably good default values, but
sometimes some fine-tuning might be required. I guess more testing is
needed to determine the smoothing behaviour of different lambda values, or
there is someone out there who understands the theory behind the code and
could help.

Markus M

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1088#comment:9&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------

Comment(by mmetz):

Replying to [comment:9 mmetz]:
> Replying to [comment:8 kyngchaos]:
> > A couple notes/questions:
> >
> > I didn't completely understand the description for the sparse option,
but it didn't look like it was needed here. Now it looks to me like it's
essentially a mask option, like the maskmap option in v.surf.rst. For
now, the mask is implied when the results are patched into the original
raster. I guess a mask option (if sparse is such) only makes sense if it
reduces processing time by ignoring the masked areas.
>
> Right. Unfortunately, using sparse points as input for v.surf.bspline
does not reduce processing time substantially, although it could be
implemented similar to what I did for r.resamp.bspline.

I just noticed that I did implement it, interpolation in v.surf.bspline is
skipped if a sparse points vector is given but no sparse points fall into
the current subregion. The advantage of the sparse points approach for
filling NULL cells would be that more than the currently 3 cells around a
gap are used for interpolation, providing (in theory) better results for
larger gaps.

Markus M

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1088#comment:10&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.5.0
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------

Comment(by neteler):

This patch (I have updated it right now to current SVN) is yet interesting
and should IMHO be submitted.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1088#comment:11&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-develbranch6
Keywords: fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------
Changes (by hamish):

  * milestone: 6.5.0 => 6.4.3

Comment:

* applied in devbr6 by MarkusN in r50114, with sie,sin = 3*res and the
default lambda_i=1.
  * applied to trunk by luca in r50128 with sie,sin = 1*res and the default
lambda_i=1.

see above comments for MarkusM's recommendation for using si=2*res and
lambda=0.005.
shall we apply those coeff's? MM: are those numbers going to be dataset
dependent, or are they generally good for the way r.fillnulls works by
extracting the centers of the 3 cell buffer pixels as the vector starting
points?

  * in 6.5svn I added a step which zooms into the minimum region containing
all holes before running v.surf.rst. For my data this made the region
about half the size and r.fillnulls ran twice as fast. beforehand
v.surf.rst and v.surf.bspline bicubic methods took about the same amount
of time (6.5svn versions..), so for my particular srtm data the v.surf.rst
method was now twice as fast as the v.surf.bspline bicubic method. I tried
running v.surf.bspline with the region zoom but it made no difference
besides using 30 subregions (with a few empty) instead of the original 77
subregions (with a lot empty). actually for bspline with the region zoom
it took about 10-15% longer, so I only applied it to the RST method, i.e.
processing time was mostly independent from region settings. I suspect the
original subregion segments were slightly more efficient than the zoomed
version due to a bit of luck of where the points were & what subregions
were empty.

both RST and bspline results were (very nearly) identical to without the
zoom.

nominating both the method= and the RST zoom patch for inclusion in 6.4.3.

trialing bspline in grass7 now... rst time was about the same as in
devbr6, a wee bit faster due to partial OpenMP support.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1088#comment:12&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-develbranch6
Keywords: r.fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------
Changes (by hamish):

  * keywords: fillnulls => r.fillnulls

Comment:

> trialing bspline in grass7 now...

the zoom trick for bspline is the same as in grass65: almost the same but
slightly slower.

I changed the bspline coeffs to si=3*res and lambda_i = 0.005. Should we
change the default option answer to that? I note in the cross-validation
code there the maximum they try is 0.05, so maybe 1.0 is way out of useful
range.

with that, time to complete was not quite as fast as zoomed-RST, but about
twice as fast as in grass65. (partly due to si= settings, partly due to
using some OpenMP, and perhaps partly due to speedups in trunk that MM was
talking about?)

> both RST and bspline results were (very nearly) identical to without the
zoom.

(there I meant r.univar results, but really that should be done on the
cells-to-be patched in, not the final result where n_patched is quite a
small percentage of the whole. an over-zoomed-in d.rast overlay didn't
show discernible visual color change)

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1088#comment:13&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-develbranch6
Keywords: r.fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------

Comment(by mmetz):

Replying to [comment:12 hamish]:
> * applied in devbr6 by MarkusN in r50114, with sie,sin = 3*res and the
default lambda_i=1.
> * applied to trunk by luca in r50128 with sie,sin = 1*res and the
default lambda_i=1.
>
> see above comments for MarkusM's recommendation for using si=2*res and
lambda=0.005.

The sie/sin settings are the result of guestimation and experimenting. I
found that for evenly placed points, e.g. a vectorized raster without
gaps, 1 * res gives nice results. With r.fillnulls, the points are not
evenly spaced and sie/sin needs to be larger than 1*res in order to avoid
artifacts. For very large gaps, support points are needed anyway by both
RST and bspline. For moderately large gaps, 2*res or 3*res seems to work
fine with bspline.

The idea of adding bspline to r.fillnulls is to have a different
interpolation method which should give different results, just like in
r.resamp.interp. RST tends to provide more detail at the cost of
overshoots, whereas bspline with method=bilinear does not produce
overshoots and produces a smoother surface. Bspline with method=bicubic
can produce results very similar to RST, but AFAIU the idea of bspline
interpolation is to avoid overshoots and to produce a smoother surface.
Therefore r.fillnulls could just as well have the method options
rst,bspline with bspline using bilinear by default. The results would then
be intentionally different.

I have modified r.fillnulls in trunk to use r.resamp.bspline which is
designed for raster interpolation instead v.surf.bspline. That avoids the
vectorization step and essentially reduces the processing to
r.resamp.bspline -n followed by r.patch.

Markus M

PS: I fixed r.buffer in trunk, it could now be used (again) by r.fillnulls
instead of r.grow.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1088#comment:14&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-develbranch6
Keywords: r.fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------

Comment(by hamish):

> Replying to [comment:12 hamish]:
> > see above comments for MarkusM's recommendation for using si=2*res and
> > lambda=0.005.
Replying to [comment:14 mmetz]:
> The sie/sin settings are the result of guestimation and experimenting.

any idea about the lambda_i setting? I suggest to change the default to
0.01, but
I've only tested with lat/lon srtm data so far. (v.surf.bspline cross-
validation
output is now a bit easier to digest in 6.5svn, btw; will port to trunk
soon)

for that data res*3.0 seemed to do well for sie,sin. res*1.0 was not
enough for
the patched tiles I tried it with.

> RST tends to provide more detail at the cost of overshoots,

fwiw in my tests yesterday v.surf.bspline bicubic with lambda=0.005 and
si?=res*3
did the best job, v.surf.bspline with default lambda=1 wasn't totally
horrible, but wasn't great, and rst was fairly obvious something had been
patched in (but I didn't tune the tension at all).

> whereas bspline with method=bilinear does not produce overshoots and
produces a
> smoother surface.

but the downside of that is that it lowers the contrast/rounds down
towards the
mean more than a spline fit would.. which is an issue because at least for
srtm the holes usually happen in the mountains. fwiw most of the overshoot
errors from r.fillnulls are from the masked area afaict, so perhaps can be
ignored.

> Bspline with method=bicubic can produce results very similar to RST, but
AFAIU the
> idea of bspline interpolation is to avoid overshoots and to produce a
smoother
> surface. Therefore r.fillnulls could just as well have the method
options
> rst,bspline with bspline using bilinear by default. The results would
then be
> intentionally different.

I personally prefer bicubic to bilinear (will let a job run slower if it
means a
better end result), and I'd vote to keep method=rst,bicubic,bilinear
instead of
method= + method2= options. the user doesn't care about which middleware
modules,
are doing the work, just the beginning and the end.

> I have modified r.fillnulls in trunk to use r.resamp.bspline which is
designed
> for raster interpolation instead v.surf.bspline.

I'll try it out & see how it does against the others.

> That avoids the vectorization step

fwiw with -z that wasn't too costly, but happy to try out a new option and
see if it
does as well as v.surf.bspline's bicubic + lambda=0.005 (which _really_
did nicely).

also, what would it take to get 'g.region vect=' to work with a level 1
map? right
now you get a no-topology error if you try it with 'r.to.vect -b' (& so
make the
rst method prep work even smaller/faster/less ram).

> and essentially reduces the processing to r.resamp.bspline -n followed
by
> r.patch.

btw, you'll need to move that del_temp_region back to where it was. with
it still
in place when you run r.patch you're cropping the patched result with the
zoomed-in
region. the python lib create temp region fn includes a on-exit hook to
automatically
unset and delete itself when the module ends, so don't worry about
leftovers.

> PS: I fixed r.buffer in trunk, it could now be used (again) by
r.fillnulls
> instead of r.grow.

you mean r.buffer.py :slight_smile: "r.buffer2" (the classic C version) in trunk
always worked bug free for many years, and is about 50x faster (race them
with `time`). Perhaps the python version was an old experiment to see if
some modules could be combined in a single engine?

thanks,
Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1088#comment:15&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-develbranch6
Keywords: r.fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------

Comment(by mmetz):

Replying to [comment:15 hamish]:
> > Replying to [comment:12 hamish]:
> > > see above comments for MarkusM's recommendation for using si=2*res
and
> > > lambda=0.005.
> Replying to [comment:14 mmetz]:
> > The sie/sin settings are the result of guestimation and experimenting.
>
> any idea about the lambda_i setting?

In my tests, 1 is usually way too high, 0.005 seems to do well in
mountainous areas, 0.1 is better for nearly flat areas.
>
> > RST tends to provide more detail at the cost of overshoots,
>
> fwiw in my tests yesterday v.surf.bspline bicubic with lambda=0.005 and
si?=res*3
> did the best job, v.surf.bspline with default lambda=1 wasn't totally
horrible, but wasn't great, and rst was fairly obvious something had been
patched in (but I didn't tune the tension at all).

I think it is not easy to avoid segmentation artifacts with RST (very high
npmin, very low segmax maybe).
>
> > whereas bspline with method=bilinear does not produce overshoots and
produces a
> > smoother surface.
>
> but the downside of that is that it lowers the contrast/rounds down
towards the
> mean more than a spline fit would..

Sometimes this is a downside, sometimes not. For hydrological analysis,
overshoots are a problem and must be dealt with somehow.
>
> > Bspline with method=bicubic can produce results very similar to RST,
but AFAIU the
> > idea of bspline interpolation is to avoid overshoots and to produce a
smoother
> > surface. Therefore r.fillnulls could just as well have the method
options
> > rst,bspline with bspline using bilinear by default. The results would
then be
> > intentionally different.
>
> I personally prefer bicubic to bilinear (will let a job run slower if it
means a
> better end result), and I'd vote to keep method=rst,bicubic,bilinear
instead of
> method= + method2= options. the user doesn't care about which middleware
modules,
> are doing the work, just the beginning and the end.

Sounds good. Anyway, the manual should also state that r.fillnulls is an
attempt for semi-automated NULL filling, and if in doubt this must be done
manually to obtain the desired results.

>
> what would it take to get 'g.region vect=' to work with a level 1 map?

The routines are already in v.info (trunk).

>
>
> > and essentially reduces the processing to r.resamp.bspline -n followed
by
> > r.patch.
>
> btw, you'll need to move that del_temp_region back to where it was. with
it still
> in place when you run r.patch you're cropping the patched result with
the zoomed-in
> region.

Oops. Changed in r50803. The grid geometry of the input raster is still
maintained, though.

>
> > PS: I fixed r.buffer in trunk, it could now be used (again) by
r.fillnulls
> > instead of r.grow.
>
> you mean r.buffer.py :slight_smile: "r.buffer2" (the classic C version) in trunk
always worked bug free for many years, and is about 50x faster (race them
with `time`). Perhaps the python version was an old experiment to see if
some modules could be combined in a single engine?

Hmm. One of the two r.buffer's should probably be deactivated, after
comparative testing with latlong and real projections.

Markus M

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1088#comment:16&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-develbranch6
Keywords: r.fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------

Comment(by hamish):

Replying to [comment:15 hamish]:
> > any idea about the lambda_i setting?
Replying to [comment:16 mmetz]:
> In my tests, 1 is usually way too high, 0.005 seems to do
> well in mountainous areas, 0.1 is better for nearly flat areas.

in trunk and devbr6 I have changed it to 0.01 now. I would suggest to
backport to 6.4svn ASAP before it harms any more data, but would like to
test it with some meter-based LIDAR data first, as I've only tried with
lat/lon srtm so far.

> I think it is not easy to avoid segmentation artifacts with RST (very
high
> npmin, very low segmax maybe).

for r.fillnulls I'd expect the 3-cell buffer of most holes not to fall on
segmentation boundaries. it wasn't that it had obvious segmentation lines
in it, rather it just didn't follow the ridges and valleys in such a
natural way. (qualitative eyeball analysis) I've been convinced for years
that there must be a way to avoid the segmentation artifacts without
massively overlapping the quadtree boxes.. still keeping an eye out.

> Sounds good. Anyway, the manual should also state that r.fillnulls
> is an attempt for semi-automated NULL filling, and if in doubt this
> must be done manually to obtain the desired results.

yeah, the man page needs quite a bit of work actually.. and I can't quite
figure out what the WARNING is trying to get you to delta against with
r.mapcalc to verify the interpolation. seems an impossible task, since if
you knew the correct answer you there'd be no reason to interpolate.

> Hmm. One of the two r.buffer's should probably be deactivated, after
> comparative testing with latlong and real projections.

I get 4.3sec for a test in r.buffer.py, 0.140 sec for r.buffer.c, for a
simple test starting at a single cell, with visually identical results.
Actually I was pleasantly surprised that r.buffer.py did the right thing
in lat/lon and made a pear-shaped buffer for the plate carree display,
wider nearer to the poles.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1088#comment:17&gt;
GRASS GIS <http://grass.osgeo.org>

#1088: r.fillnulls: support other interpolation methods
-------------------------+--------------------------------------------------
Reporter: kyngchaos | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 6.4.3
Component: Raster | Version: svn-develbranch6
Keywords: r.fillnulls | Platform: All
      Cpu: All |
-------------------------+--------------------------------------------------

Comment(by mmetz):

Replying to [comment:17 hamish]:
> Replying to [comment:15 hamish]:
> > > any idea about the lambda_i setting?
> Replying to [comment:16 mmetz]:
> > In my tests, 1 is usually way too high, 0.005 seems to do
> > well in mountainous areas, 0.1 is better for nearly flat areas.
>
> in trunk and devbr6 I have changed it to 0.01 now. I would suggest to
backport to 6.4svn ASAP before it harms any more data, but would like to
test it with some meter-based LIDAR data first, as I've only tried with
lat/lon srtm so far.

Tested with LiDAR datasets, lambda=1 is too high, synced in 6.4 to
trunk/devbr6 with default lambda=0.01.
>
>
>
> > I think it is not easy to avoid segmentation artifacts with RST (very
high
> > npmin, very low segmax maybe).
>
> for r.fillnulls I'd expect the 3-cell buffer of most holes not to fall
on segmentation boundaries. it wasn't that it had obvious segmentation
lines in it, rather it just didn't follow the ridges and valleys in such a
natural way. (qualitative eyeball analysis) I've been convinced for years
that there must be a way to avoid the segmentation artifacts without
massively overlapping the quadtree boxes.. still keeping an eye out.

I had obvious segmentation lines with RST. From my testing with the
bspline method I think that there is no way around massively overlapping
quadtree boxes.
>
>
> > Hmm. One of the two r.buffer's should probably be deactivated, after
> > comparative testing with latlong and real projections.
>
> I get 4.3sec for a test in r.buffer.py, 0.140 sec for r.buffer.c, for a
simple test starting at a single cell, with visually identical results.
Actually I was pleasantly surprised that r.buffer.py did the right thing
in lat/lon and made a pear-shaped buffer for the plate carree display,
wider nearer to the poles.

In trunk, r.buffer.c loads both input and output completely to memory,
whereas r.buffer.py using r.grow.distance doesn't. Therefore r.buffer.c is
faster for smaller maps but will freeze the system if it goes into swap
space. r.buffer.py keeps the disk busy, but the system remains responsive.

Markus M

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/1088#comment:18&gt;
GRASS GIS <http://grass.osgeo.org>