[GRASS-dev] [GRASS GIS] #1938: r.fillnulls: per hole filling speed-up

#1938: r.fillnulls: per hole filling speed-up
---------------------------+------------------------------------------------
Reporter: sbl | Owner: grass-dev@…
     Type: enhancement | Status: new
Priority: normal | Milestone: 7.0.0
Component: Shell Scripts | Version: svn-trunk
Keywords: r.fillnulls | Platform: Unspecified
      Cpu: Unspecified |
---------------------------+------------------------------------------------
When running on a large grid (60,000 x 50,000 cells) with lots of NoData
areas (52,000 holes) r.fillnulls is very slow.
I figured out, that there was significant potential for speedup in the way
r.fillnulls applies region cropping (the way it moves the region from hole
to hole).

Therefore I would like to propose a speedup provided by the attached
patch.

Main difference is, that the raster based region cropping (which ran a map
calculator expression on (nearly) the entire input raster) within the loop
over the holes is replaced by a vector based approach (similar to the one
used in r.connectivity.distance).
Here, before r.fillnulls loops over the holes, the latter are converted to
vector (r.to.vect). Within the loop the “focal hole” is extracted based on
category (v.extract) and the region is adjusted to this single polygon.
By doing so, the r.mapcalc function for raster based region cropping could
be replaced by the (significantly faster v.extract).
Finally, if I am not mistaken the two runs of g.region could be joined to
one (though this should not have major impact on the processing speed…).

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1938&gt;
GRASS GIS <http://grass.osgeo.org>

#1938: r.fillnulls: per hole filling speed-up
----------------------------+-----------------------------------------------
  Reporter: sbl | Owner: grass-dev@…
      Type: enhancement | Status: closed
  Priority: normal | Milestone: 7.0.0
Component: Shell Scripts | Version: svn-trunk
Resolution: fixed | Keywords: r.fillnulls
  Platform: Unspecified | Cpu: Unspecified
----------------------------+-----------------------------------------------
Changes (by marisn):

  * status: new => closed
  * resolution: => fixed

Comment:

A slightly modified version has been applied in r57030

Thanks, Stefan!

Next speed up for r.fillnulls would be if someone would implement parallel
filling of holes.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1938#comment:1&gt;
GRASS GIS <http://grass.osgeo.org>

#1938: r.fillnulls: per hole filling speed-up
--------------------------+-------------------------------------------------
  Reporter: sbl | Owner: grass-dev@…
      Type: enhancement | Status: reopened
  Priority: normal | Milestone: 7.1.0
Component: Python | Version: svn-trunk
Resolution: | Keywords: r.fillnulls
  Platform: All | Cpu: All
--------------------------+-------------------------------------------------
Changes (by sbl):

  * status: closed => reopened
  * component: Shell Scripts => Python
  * platform: Unspecified => All
  * milestone: 7.0.0 => 7.1.0
  * resolution: fixed =>
  * cpu: Unspecified => All

Comment:

Please find attached another proposal for speed-up of rst-interpolation by
hole.

In the diff, there are two sections which can probably be removed in
addition:

1) The new r.clump version should account for NoData, so the following
r.mapcalc operation can be removed (could not test it since I work with an
older version).

2) I was unsure if r.patch has a limit regarding the number of open maps,
if not than block-wise application of r.patch can be removed too...

Maybe better to use bilinear or bicubic interpolation as default as it is
significant faster (and simpler) than rst?

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/1938#comment:2&gt;
GRASS GIS <http://grass.osgeo.org>