[GRASS-dev] [GRASS GIS] #398: r.watershed with MFD

GDAL · December 12, 2008, 11:34am

#398: r.watershed with MFD
-------------------------+--------------------------------------------------
Reporter: mmetz | Owner: grass-dev@lists.osgeo.org
Type: enhancement | Status: new
Priority: minor | Milestone: 7.0.0
Component: Raster | Version: svn-trunk
Keywords: | Platform: Unspecified
Cpu: Unspecified |
-------------------------+--------------------------------------------------
I open a new ticket for easier tracking of all the bug reports to come...

With the attached patch I want to submit the last (for the time being)
modifications to r.watershed. This patch fixes a bug in flag
initialization, also present in the original (pre-MFD and pre-faster-
search) version. Further on, streams and basins are now reasonably well
(IMO) determined for MFD.

Flow accumulation with SFD (single flow direction) is unchanged apart from
flow accumulation output now being DCELL instead of CELL.

Flow accumulation with MFD (multiple flow direction) is supported and
enabled by default to keep the user interface similar to r.terraflow.

For easy DEMs, e.g. <elevation> in the North Carolina dataset, the results
are similar. Differences emerge where MFD kicks in and the flow diverges.
Generally, more smaller basins are created with MFD than with SFD keeping
the same basin threshold, because diverging flow leads to more cells with
flow accumulation above the given threshold.

For difficult DEMs, e.g. <elev_lid972_1m> in the North Carolina dataset,
the results are very different. MFD output seems to be more realistic.
Here, more smaller basins with unnaturally straight borders are created
with SFD. Basins created with MFD seem more realistic and have naturally
looking borders, some small basins are created, but no sliver basins like
in SFD. Stream segments describe the center of broad streams pretty well,
without branching off inside a broad stream.

The results are not perfect, but usable, I think. These are my personal
assessments and other people, you, should have a close look at the MFD
results and please report any bugs.

Have fun playing around with the convergence factor!

Best,

Markus

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/398>
GRASS GIS <http://grass.osgeo.org>

GDAL · December 12, 2008, 1:23pm

#398: r.watershed with MFD
--------------------------+-------------------------------------------------
  Reporter: mmetz | Owner: grass-dev@lists.osgeo.org
      Type: enhancement | Status: new
  Priority: minor | Milestone: 7.0.0
Component: Raster | Version: svn-trunk
Resolution: | Keywords:
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by glynn):

Replying to [ticket:398 mmetz]:

> With the attached patch I want to submit the last (for the time being)
modifications to r.watershed.

Applied as r34833.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/398#comment:1>
GRASS GIS <http://grass.osgeo.org>

GDAL · December 13, 2008, 5:31am

#398: r.watershed with MFD
--------------------------+-------------------------------------------------
  Reporter: mmetz | Owner: grass-dev@lists.osgeo.org
      Type: enhancement | Status: new
  Priority: minor | Milestone: 7.0.0
Component: Raster | Version: svn-trunk
Resolution: | Keywords:
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by hamish):

Replying to [ticket:398 mmetz]:
> With the attached patch I want to submit the last (for the time being)
modifications to r.watershed. This patch fixes a bug in flag
initialization, also present in the original (pre-MFD and pre-faster-
search) version.

Hi,

could you isolate that bugfix, i.e. what needs to be backported to 6.4?

> Flow accumulation with SFD (single flow direction) is unchanged
> apart from flow accumulation output now being DCELL instead of
> CELL.

nice.

> Flow accumulation with MFD (multiple flow direction) is supported
> and enabled by default to keep the user interface similar to
> r.terraflow.

ok, if there is a major change like that in grass7 it is worth
noting clearly in the help page. also the help page needs updating
to reflect that it is no longer "much slower than r.terraflow" and
the parts referring to the A^T search algorithm, and any references.
I would guess to leave Ehlschlaeger's paper there for the
r.watershed ref even if the search algorithm is replaced.

> Have fun playing around with the convergence factor!

...

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/398#comment:2>
GRASS GIS <http://grass.osgeo.org>

GDAL · December 13, 2008, 9:20am

#398: r.watershed with MFD
--------------------------+-------------------------------------------------
  Reporter: mmetz | Owner: grass-dev@lists.osgeo.org
      Type: enhancement | Status: new
  Priority: minor | Milestone: 7.0.0
Component: Raster | Version: svn-trunk
Resolution: | Keywords:
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by mmetz):

Replying to [comment:2 hamish]:
> Replying to [ticket:398 mmetz]:
> > With the attached patch I want to submit the last (for the time being)
modifications to r.watershed. This patch fixes a bug in flag
initialization, also present in the original (pre-MFD and pre-faster-
search) version.
>
> could you isolate that bugfix, i.e. what needs to be backported to 6.4?
>
Attached as r.watershed.bugfix.grass64.patch
>
> > Flow accumulation with SFD (single flow direction) is unchanged
> > apart from flow accumulation output now being DCELL instead of
> > CELL.
>
> nice.
>
Also the code for SFD is unchanged apart from flow accumulation output now
being DCELL instead of CELL, maybe I have to emphasize that.

> > Flow accumulation with MFD (multiple flow direction) is supported
> > and enabled by default to keep the user interface similar to
> > r.terraflow.
>
> ok, if there is a major change like that in grass7 it is worth
> noting clearly in the help page. also the help page needs updating
> to reflect that it is no longer "much slower than r.terraflow"

Done in attached patch r.watershed.manpage.patch.

> the parts referring to the A^T^ search algorithm, and any references.
> I would guess to leave Ehlschlaeger's paper there for the
> r.watershed ref even if the search algorithm is replaced.
>
The A^T^ search algorithm is untouched and in place and also used for MFD.
I have already included a reference for MFD, but the actual implementation
is a hybrid of MFD and the flow path determined with the A^T^ search
algorithm. The A^T^ path, the result of the A^T^ search, is the guidance
around which flow is distributed in MFD. Further on, MFD can't get out of
depressions and over obstacles, but the A^T^ path can and still does in
MFD. Chuck Ehlschlaeger's paper must stay because also with MFD the whole
concept of r.watershed is based on Chuck Ehlschlaeger's paper. The only
part that has been replaced is the sorting method for the A^T^ search to
speed up the analysis, the A^T^ search itself works as before. For MFD, I
have not replaced anything, I have only added code, ok, modified existing
code to switch between SFD and MFD, but nothing else.

The attached patch named r.watershed.manpage.patch updates the manpage
with your suggestions if not already present.

Markus

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/398#comment:3>
GRASS GIS <http://grass.osgeo.org>

GDAL · December 15, 2008, 3:50am

#398: r.watershed with MFD
--------------------------+-------------------------------------------------
  Reporter: mmetz | Owner: grass-dev@lists.osgeo.org
      Type: enhancement | Status: closed
  Priority: minor | Milestone: 7.0.0
Component: Raster | Version: svn-trunk
Resolution: fixed | Keywords:
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Changes (by hamish):

* status: new => closed
* resolution: => fixed

Comment:

thanks, all patches applied/merged.

Hamish

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/398#comment:4>
GRASS GIS <http://grass.osgeo.org>

GDAL · December 20, 2008, 9:27am

#398: r.watershed with MFD
--------------------------+-------------------------------------------------
  Reporter: mmetz | Owner: grass-dev@lists.osgeo.org
      Type: enhancement | Status: reopened
  Priority: minor | Milestone: 7.0.0
Component: Raster | Version: svn-trunk
Resolution: | Keywords:
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Changes (by mmetz):

* status: closed => reopened
* resolution: fixed =>

Comment:

the two attached patches

r.watershed.64.length.slope.patch

and

r.watershed.70.length.slope.patch

check whether a basin threshold has been specified when length.slope
(length_slope in 70) is specified as output. This has been a requirement
in the code before, but wasn't enforced and reported to the user.

Markus M

PS: Please leave the ticket open, more fixes to come for MFD mode.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/398#comment:5>
GRASS GIS <http://grass.osgeo.org>

GDAL · January 7, 2009, 11:33am

#398: r.watershed with MFD
--------------------------+-------------------------------------------------
  Reporter: mmetz | Owner: grass-dev@lists.osgeo.org
      Type: enhancement | Status: reopened
  Priority: minor | Milestone: 7.0.0
Component: Raster | Version: svn-trunk
Resolution: | Keywords:
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by mmetz):

A new patch against trunk r35238 is attached, named
r.watershed.20090107.patch

This patch includes the following changes:

- MFD flow accumulation is slightly improved.

- MFD streams and basins are now equal to or better than SFD streams and
basins (they look more natural, no slivers or rectangles). Striking
differences are visible with elev_lid792_1m (multiplied by 1000 to get mm
accuracy) in nc_spm_08.

- Segmented mode is slightly faster, more noticeable with larger regions.

- In segmented mode, the dseg routines for DCELL output maps were actually
writing out CELL maps, not DCELL maps, changed to DCELL maps. DCELL_TYPE
is required for LS and S factor for USLE and for flow accumulation.

- Improved color rules for flow accumulation based on standard deviation
and log transformation, visual output would be obsolete now. BTW, in
visual output all cells with negative flow accumulation (offmap inflow)
are set to zero. That's an old, existing feature.

- In segmented mode, G_percent() in length slope determination is now
counting up, no longer counting down (was a FIXME).

Can you developers please test this patch? Thanks!

Happy New Year,

Markus M

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/398#comment:6>
GRASS GIS <http://grass.osgeo.org>

GDAL · January 15, 2009, 9:39am

#398: r.watershed with MFD
--------------------------+-------------------------------------------------
  Reporter: mmetz | Owner: grass-dev@lists.osgeo.org
      Type: enhancement | Status: reopened
  Priority: minor | Milestone: 7.0.0
Component: Raster | Version: svn-trunk
Resolution: | Keywords:
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by mmetz):

I have submitted a new version of r.watershed to trunk with various
changes

MFD is fully supported: drainage directions, stream segments, basins,
half-basins, RUSLE LS and S factors are all in accordance with MFD flow
accumulation.

The module is slightly faster and uses a bit less memory. See updated
documentation to estimate memory requirements for the ram (all in memory)
mode (max 31 MB of RAM for 1 million cells).

It is no longer necessary to create a MASK for cells with unknown
elevation (NULL values). All cells with unknown elevation (masked or NULL
values) are skipped. Zero and negative values are valid elevation values.

The color rules generated by the module for flow accumulation have worked
for all tests so far and allow visual inspection without manually
assigning a color table. I'm using standard deviation and log transform,
based on Hamish's (I think) suggestions. Visual output is obsolete but
left for comparison.

The output drainage direction is slightly changed in case an input map
with real depressions was supplied. Previously, real depressions had a
drainage direction of -1, but -1 was and is a valid drainage direction
indicating a drainage of 45 degree CCW from East towards a cell with
unknown elevation. Therefore real depressions have now a drainage
direction of 0. As before, if a cell drains due East the drainage
direction is 360 degrees, 0 (zero) as drainage direction was unused.

RUSLE LS and S factor output maps are now of type DCELL without
multiplication by 100. Both factors have always been processed as type
double, previously the output maps for the two factors were written out by
typecasting from (DCELL * 100) to CELL.

Please read the updated documentation before using the new module.

Best regards,

Markus M

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/398#comment:7>
GRASS GIS <http://grass.osgeo.org>

GDAL · February 9, 2009, 9:14am

#398: r.watershed with MFD
--------------------------+-------------------------------------------------
  Reporter: mmetz | Owner: grass-dev@lists.osgeo.org
      Type: enhancement | Status: reopened
  Priority: minor | Milestone: 7.0.0
Component: Raster | Version: svn-trunk
Resolution: | Keywords:
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by mmetz):

Helena Mitasova and Michael Barton asked previously about backporting
r.watershed with MFD to grass6. Now that develbranch_6 stays as version
6.5.svn, does someone have objections against backporting r.watershed from
trunk to develbranch_6?

It would be backported with the following modifications for backwards
compatibility:

disable MFD by default, it will be enabled with a new -f flag. I don't
want to (ab)use the -s flag of trunk because it has the opposite meaning.
The -s flag will be removed.

RUSLE LS and S factors will again be multiplied by 100, as before. The
output raster type of the LS and S factors has to be changed to DCELL
though, otherwise MFD won't work, because flow accumulation needs to be of
type DCELL for MFD, i.e. always, and the segmented mode uses the same
function for writing out any variable of type DCELL. Previously, LS and S
factor were stored as type DCELL, but both in segmented mode and in ram
mode written out as type (CELL) (factor * 100). This change from CELL to
DCELL can not be reverted, at the same time supporting flow accumulation
output as type DCELL, without a lot of rewriting, actually more rewriting
than was necessary to implement MFD support. The change from CELL to DCELL
for flow accumulation and the RUSLE factors will hopefully not break
scripts.

I would need some help on how to properly backport the various changes
from trunk to devbr6. According to the svn howto, svn merge backports one
particular change if I got that right.

Markus M

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/398#comment:8>
GRASS GIS <http://grass.osgeo.org>

GDAL · March 1, 2009, 2:09pm

#398: r.watershed with MFD
--------------------------+-------------------------------------------------
  Reporter: mmetz | Owner: grass-dev@lists.osgeo.org
      Type: enhancement | Status: closed
  Priority: minor | Milestone: 7.0.0
Component: Raster | Version: svn-trunk
Resolution: worksforme | Keywords:
  Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Changes (by mmetz):

* status: reopened => closed
* resolution: => worksforme

Comment:

backported to debr6 in r35905 with full backwards compatibility, closing
ticket.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/398#comment:9>
GRASS GIS <http://grass.osgeo.org>