[GRASS-dev] Re: grass-dev Digest, Vol 33, Issue 37

On Jan 15, 2009, at 5:22 AM, <grass-dev-request@lists.osgeo.org> wrote:

Date: Thu, 15 Jan 2009 09:39:00 -0000
From: "GRASS GIS" <trac@osgeo.org>
Subject: [GRASS-dev] Re: [GRASS GIS] #398: r.watershed with MFD
To: undisclosed-recipients:;
Message-ID: <048.b6a7364e017fd287d20f06fe708aedbe@osgeo.org>
Content-Type: text/plain; charset="utf-8"

#398: r.watershed with MFD
--------------------------+-------------------------------------------------
Reporter: mmetz | Owner: grass-dev@lists.osgeo.org
     Type: enhancement | Status: reopened
Priority: minor | Milestone: 7.0.0
Component: Raster | Version: svn-trunk
Resolution: | Keywords:
Platform: Unspecified | Cpu: Unspecified
--------------------------+-------------------------------------------------
Comment (by mmetz):

I have submitted a new version of r.watershed to trunk with various
changes

MFD is fully supported: drainage directions, stream segments, basins,
half-basins, RUSLE LS and S factors are all in accordance with MFD flow
accumulation.

The module is slightly faster and uses a bit less memory. See updated
documentation to estimate memory requirements for the ram (all in memory)
mode (max 31 MB of RAM for 1 million cells).

It is no longer necessary to create a MASK for cells with unknown
elevation (NULL values). All cells with unknown elevation (masked or NULL
values) are skipped. Zero and negative values are valid elevation values.

The color rules generated by the module for flow accumulation have worked
for all tests so far and allow visual inspection without manually
assigning a color table. I'm using standard deviation and log transform,
based on Hamish's (I think) suggestions. Visual output is obsolete but
left for comparison.

The output drainage direction is slightly changed in case an input map
with real depressions was supplied. Previously, real depressions had a
drainage direction of -1, but -1 was and is a valid drainage direction
indicating a drainage of 45 degree CCW from East towards a cell with
unknown elevation. Therefore real depressions have now a drainage
direction of 0. As before, if a cell drains due East the drainage
direction is 360 degrees, 0 (zero) as drainage direction was unused.

RUSLE LS and S factor output maps are now of type DCELL without
multiplication by 100. Both factors have always been processed as type
double, previously the output maps for the two factors were written out by
typecasting from (DCELL * 100) to CELL.

Please read the updated documentation before using the new module.

Best regards,

Markus,

Can this go into develbranch_6 too?

Michael

Michael Barton wrote:

On Jan 15, 2009, at 5:22 AM, <grass-dev-request@lists.osgeo.org> wrote:

[...]

Comment (by mmetz):

I have submitted a new version of r.watershed to trunk with various
changes

[...]

Markus,

Can this go into develbranch_6 too?

I hope so:-) But I would like to get the new functionality tested by others before it is backported in case I missed something. You mentioned that you are testing the new functionality, and you are using Mac while I use Linux 64bit, so if the new version runs fine and produces proper results on your system that would be encouraging!
Below are test scripts for the North Carolina sample dataset nc_spm_08 and the Spearfish dataset spearfish60. You can copy the commands for each dataset into a script and then run the script, after updating to today's version of r.watershed:-)

Markus M

Testing commands for the elevation map elev_lid792_1m in the North Carolina sample dataset nc_spm_08. SFD and MFD output is strikingly different.

g.region rast=elev_lid792_1m
# multiply elev_lid792_1m to get millimetres as vertical units
r.mapcalc "elev_lid792_1m.1mm = round(elev_lid792_1m * 1000.0)"

# SFD (D8) flow distribution
r.watershed elev=elev_lid792_1m.1mm threshold=3000 \
accum=elev_lid792_1m.1mm.acc.sfd drain=elev_lid792_1m.1mm.drain.sfd \
basin=elev_lid792_1m.1mm.basin.sfd stream=elev_lid792_1m.1mm.stream.sfd \
half_b=elev_lid792_1m.1mm.halfb.sfd vis=elev_lid792_1m.1mm.viz.sfd \
length_sl=elev_lid792_1m.1mm.ls.sfd slope_st=elev_lid792_1m.1mm.sls.sfd -s

# MFD flow distribution
r.watershed elev=elev_lid792_1m.1mm threshold=3000 \
accum=elev_lid792_1m.1mm.acc.mfd drain=elev_lid792_1m.1mm.drain.mfd \
basin=elev_lid792_1m.1mm.basin.mfd stream=elev_lid792_1m.1mm.stream.mfd \
half_b=elev_lid792_1m.1mm.halfb.mfd vis=elev_lid792_1m.1mm.viz.mfd \
length_sl=elev_lid792_1m.1mm.ls.mfd slope_st=elev_lid792_1m.1mm.sls.mfd \
convergence=5

Testing commands for the elevation map elevation.10m in the Spearfish sample dataset spearfish60. Differences between SFD and MFD are mainly in the northern half.

g.region rast=elevation.10m
# multiply elevation.10m to get centimetres as vertical units
r.mapcalc "elevation.10m.1cm = round(elevation.10m * 100.0)"

# SFD (D8) flow distribution
r.watershed elev=elevation.10m.1cm threshold=3000 \
accum=elevation.10m.1cm.acc.sfd drain=elevation.10m.1cm.drain.sfd \
basin=elevation.10m.1cm.basin.sfd stream=elevation.10m.1cm.stream.sfd \
half_b=elevation.10m.1cm.halfb.sfd vis=elevation.10m.1cm.viz.sfd \
length_sl=elevation.10m.1cm.ls.sfd slope_st=elevation.10m.1cm.sls.sfd -s

# MFD flow distribution
r.watershed elev=elevation.10m.1cm threshold=3000 \
accum=elevation.10m.1cm.acc.mfd drain=elevation.10m.1cm.drain.mfd \
basin=elevation.10m.1cm.basin.mfd stream=elevation.10m.1cm.stream.mfd \
half_b=elevation.10m.1cm.halfb.mfd vis=elevation.10m.1cm.viz.mfd \
length_sl=elevation.10m.1cm.ls.mfd slope_st=elevation.10m.1cm.sls.mfd \
convergence=5

Michael
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Michael Barton wrote:

On Jan 15, 2009, at 5:22 AM, <grass-dev-request@lists.osgeo.org> wrote:

[...]

Comment (by mmetz):

I have submitted a new version of r.watershed to trunk with various
changes

[..]

Markus,

Can this go into develbranch_6 too?

According to Markus Neteler, not as long as develbranch_6 is 6.4. If the version of develbranch_6 is changed to 6.5, I can think about asking to backport again. I will however continue to do active development in grass7. What I can do is to give instructions on how to add r.watershed from grass7 to grass6 manually on a local copy so that a subsequent svn up for grass6 doesn't create a mess. Of course this would be purely for testing purposes.

Markus M

Michael
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Markus,

I compiled last night and was able to try this on my Mac this evening. No problem at all with the North Carolina elev_lid792_1m DEM. Everything ran very fast, of course--even though it runs at 32bit on the Mac.

As expected, the MFD results look much more realistic than the SFD results. Much less linearity and more sinuosity. I can post the displays if anyone wants to see them.

No errors from the command.

No errors in compiling.

Seems to work fine on the Mac.

I'm not sure why this can't be backported to develbranch_6 since 6.4RCx is already out. Does it change any of the r.watershed arguments or behavior using the pre-MFD arguments?

Michael

On Jan 15, 2009, at 12:39 PM, Markus Metz wrote:

Michael Barton wrote:

On Jan 15, 2009, at 5:22 AM, <grass-dev-request@lists.osgeo.org> wrote:

[...]

Comment (by mmetz):

I have submitted a new version of r.watershed to trunk with various
changes

[...]

Markus,

Can this go into develbranch_6 too?

I hope so:-) But I would like to get the new functionality tested by others before it is backported in case I missed something. You mentioned that you are testing the new functionality, and you are using Mac while I use Linux 64bit, so if the new version runs fine and produces proper results on your system that would be encouraging!
Below are test scripts for the North Carolina sample dataset nc_spm_08 and the Spearfish dataset spearfish60. You can copy the commands for each dataset into a script and then run the script, after updating to today's version of r.watershed:-)

Markus M

Testing commands for the elevation map elev_lid792_1m in the North Carolina sample dataset nc_spm_08. SFD and MFD output is strikingly different.

g.region rast=elev_lid792_1m
# multiply elev_lid792_1m to get millimetres as vertical units
r.mapcalc "elev_lid792_1m.1mm = round(elev_lid792_1m * 1000.0)"

# SFD (D8) flow distribution
r.watershed elev=elev_lid792_1m.1mm threshold=3000 \
accum=elev_lid792_1m.1mm.acc.sfd drain=elev_lid792_1m.1mm.drain.sfd \
basin=elev_lid792_1m.1mm.basin.sfd stream=elev_lid792_1m.1mm.stream.sfd \
half_b=elev_lid792_1m.1mm.halfb.sfd vis=elev_lid792_1m.1mm.viz.sfd \
length_sl=elev_lid792_1m.1mm.ls.sfd slope_st=elev_lid792_1m.1mm.sls.sfd -s

# MFD flow distribution
r.watershed elev=elev_lid792_1m.1mm threshold=3000 \
accum=elev_lid792_1m.1mm.acc.mfd drain=elev_lid792_1m.1mm.drain.mfd \
basin=elev_lid792_1m.1mm.basin.mfd stream=elev_lid792_1m.1mm.stream.mfd \
half_b=elev_lid792_1m.1mm.halfb.mfd vis=elev_lid792_1m.1mm.viz.mfd \
length_sl=elev_lid792_1m.1mm.ls.mfd slope_st=elev_lid792_1m.1mm.sls.mfd \
convergence=5

Testing commands for the elevation map elevation.10m in the Spearfish sample dataset spearfish60. Differences between SFD and MFD are mainly in the northern half.

g.region rast=elevation.10m
# multiply elevation.10m to get centimetres as vertical units
r.mapcalc "elevation.10m.1cm = round(elevation.10m * 100.0)"

# SFD (D8) flow distribution
r.watershed elev=elevation.10m.1cm threshold=3000 \
accum=elevation.10m.1cm.acc.sfd drain=elevation.10m.1cm.drain.sfd \
basin=elevation.10m.1cm.basin.sfd stream=elevation.10m.1cm.stream.sfd \
half_b=elevation.10m.1cm.halfb.sfd vis=elevation.10m.1cm.viz.sfd \
length_sl=elevation.10m.1cm.ls.sfd slope_st=elevation.10m.1cm.sls.sfd -s

# MFD flow distribution
r.watershed elev=elevation.10m.1cm threshold=3000 \
accum=elevation.10m.1cm.acc.mfd drain=elevation.10m.1cm.drain.mfd \
basin=elevation.10m.1cm.basin.mfd stream=elevation.10m.1cm.stream.mfd \
half_b=elevation.10m.1cm.halfb.mfd vis=elevation.10m.1cm.viz.mfd \
length_sl=elevation.10m.1cm.ls.mfd slope_st=elevation.10m.1cm.sls.mfd \
convergence=5

Michael
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
grass-dev Info Page

On Jan 17, 2009, at 12:07 AM, Michael Barton wrote:

Markus,

I compiled last night and was able to try this on my Mac this evening. No problem at all with the North Carolina elev_lid792_1m DEM. Everything ran very fast, of course--even though it runs at 32bit on the Mac.

As expected, the MFD results look much more realistic than the SFD results. Much less linearity and more sinuosity. I can post the displays if anyone wants to see them.

No errors from the command.

No errors in compiling.

Seems to work fine on the Mac.

I'm not sure why this can't be backported to develbranch_6 since 6.4RCx is already out. Does it change any of the r.watershed arguments or behavior using the pre-MFD arguments?

Michael
- it needs to be thoroughly tested on wide range of data
before replacing it. I am trying to get to it too and Markus M has done already quite a bit of testing himself. If you can ask Isaac and/or whoever has grass7 and is around to test it with different data
(wide range of resolutions, data sources, combinations of parameters, integer or float DEMs, latlong, different size and type of depressions etc.) that would help.

Has anybody any other issues with the new r.watershed than testing?

Helena

Michael

On Jan 15, 2009, at 12:39 PM, Markus Metz wrote:

Michael Barton wrote:

On Jan 15, 2009, at 5:22 AM, <grass-dev-request@lists.osgeo.org> wrote:

[...]

Comment (by mmetz):

I have submitted a new version of r.watershed to trunk with various
changes

[...]

Markus,

Can this go into develbranch_6 too?

I hope so:-) But I would like to get the new functionality tested by others before it is backported in case I missed something. You mentioned that you are testing the new functionality, and you are using Mac while I use Linux 64bit, so if the new version runs fine and produces proper results on your system that would be encouraging!
Below are test scripts for the North Carolina sample dataset nc_spm_08 and the Spearfish dataset spearfish60. You can copy the commands for each dataset into a script and then run the script, after updating to today's version of r.watershed:-)

Markus M

Testing commands for the elevation map elev_lid792_1m in the North Carolina sample dataset nc_spm_08. SFD and MFD output is strikingly different.

g.region rast=elev_lid792_1m
# multiply elev_lid792_1m to get millimetres as vertical units
r.mapcalc "elev_lid792_1m.1mm = round(elev_lid792_1m * 1000.0)"

# SFD (D8) flow distribution
r.watershed elev=elev_lid792_1m.1mm threshold=3000 \
accum=elev_lid792_1m.1mm.acc.sfd drain=elev_lid792_1m.1mm.drain.sfd \
basin=elev_lid792_1m.1mm.basin.sfd stream=elev_lid792_1m.1mm.stream.sfd \
half_b=elev_lid792_1m.1mm.halfb.sfd vis=elev_lid792_1m.1mm.viz.sfd \
length_sl=elev_lid792_1m.1mm.ls.sfd slope_st=elev_lid792_1m.1mm.sls.sfd -s

# MFD flow distribution
r.watershed elev=elev_lid792_1m.1mm threshold=3000 \
accum=elev_lid792_1m.1mm.acc.mfd drain=elev_lid792_1m.1mm.drain.mfd \
basin=elev_lid792_1m.1mm.basin.mfd stream=elev_lid792_1m.1mm.stream.mfd \
half_b=elev_lid792_1m.1mm.halfb.mfd vis=elev_lid792_1m.1mm.viz.mfd \
length_sl=elev_lid792_1m.1mm.ls.mfd slope_st=elev_lid792_1m.1mm.sls.mfd \
convergence=5

Testing commands for the elevation map elevation.10m in the Spearfish sample dataset spearfish60. Differences between SFD and MFD are mainly in the northern half.

g.region rast=elevation.10m
# multiply elevation.10m to get centimetres as vertical units
r.mapcalc "elevation.10m.1cm = round(elevation.10m * 100.0)"

# SFD (D8) flow distribution
r.watershed elev=elevation.10m.1cm threshold=3000 \
accum=elevation.10m.1cm.acc.sfd drain=elevation.10m.1cm.drain.sfd \
basin=elevation.10m.1cm.basin.sfd stream=elevation.10m.1cm.stream.sfd \
half_b=elevation.10m.1cm.halfb.sfd vis=elevation.10m.1cm.viz.sfd \
length_sl=elevation.10m.1cm.ls.sfd slope_st=elevation.10m.1cm.sls.sfd -s

# MFD flow distribution
r.watershed elev=elevation.10m.1cm threshold=3000 \
accum=elevation.10m.1cm.acc.mfd drain=elevation.10m.1cm.drain.mfd \
basin=elevation.10m.1cm.basin.mfd stream=elevation.10m.1cm.stream.mfd \
half_b=elevation.10m.1cm.halfb.mfd vis=elevation.10m.1cm.viz.mfd \
length_sl=elevation.10m.1cm.ls.mfd slope_st=elevation.10m.1cm.sls.mfd \
convergence=5

Michael
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

just a quick note from the first run - it may be better to have SFD as default so that the same command as in grass63
gives the same result. E.g. running
g.region rast=elevation
r.watershed elevation thresh=10000 accum=accum_10K2 drain=draindir_10K2 basin=basin_10K2 stream=rivers2

gives an MFD result that some people may consider weird when compared with the old r.watershed
(although it is perfectly fine as MFD result)

running the same with r.watershed -s gives the same, more "normal" looking result.
So people who have r.watershed in scripts and expect/need SFD result would need to add -s,
or will be puzzled by the different result.

So whoever has some examples, scripts with r.watershed please run it to see whether
it behaves as expected.

Helena

P.S. I often do grass stuff from home over the net. For grass7 display through gui, even clicking off or changing
the names of layers is rather time consuming compared to d.rast with x0 (I have cable).
Any hints, tricks on how to display maps fast when testing would be appreciated.

On Jan 17, 2009, at 12:07 AM, Michael Barton wrote:

Markus,

I compiled last night and was able to try this on my Mac this evening. No problem at all with the North Carolina elev_lid792_1m DEM. Everything ran very fast, of course--even though it runs at 32bit on the Mac.

As expected, the MFD results look much more realistic than the SFD results. Much less linearity and more sinuosity. I can post the displays if anyone wants to see them.

No errors from the command.

No errors in compiling.

Seems to work fine on the Mac.

I'm not sure why this can't be backported to develbranch_6 since 6.4RCx is already out. Does it change any of the r.watershed arguments or behavior using the pre-MFD arguments?

Michael

On Jan 15, 2009, at 12:39 PM, Markus Metz wrote:

Michael Barton wrote:

On Jan 15, 2009, at 5:22 AM, <grass-dev-request@lists.osgeo.org> wrote:

[...]

Comment (by mmetz):

I have submitted a new version of r.watershed to trunk with various
changes

[...]

Markus,

Can this go into develbranch_6 too?

I hope so:-) But I would like to get the new functionality tested by others before it is backported in case I missed something. You mentioned that you are testing the new functionality, and you are using Mac while I use Linux 64bit, so if the new version runs fine and produces proper results on your system that would be encouraging!
Below are test scripts for the North Carolina sample dataset nc_spm_08 and the Spearfish dataset spearfish60. You can copy the commands for each dataset into a script and then run the script, after updating to today's version of r.watershed:-)

Markus M

Testing commands for the elevation map elev_lid792_1m in the North Carolina sample dataset nc_spm_08. SFD and MFD output is strikingly different.

g.region rast=elev_lid792_1m
# multiply elev_lid792_1m to get millimetres as vertical units
r.mapcalc "elev_lid792_1m.1mm = round(elev_lid792_1m * 1000.0)"

# SFD (D8) flow distribution
r.watershed elev=elev_lid792_1m.1mm threshold=3000 \
accum=elev_lid792_1m.1mm.acc.sfd drain=elev_lid792_1m.1mm.drain.sfd \
basin=elev_lid792_1m.1mm.basin.sfd stream=elev_lid792_1m.1mm.stream.sfd \
half_b=elev_lid792_1m.1mm.halfb.sfd vis=elev_lid792_1m.1mm.viz.sfd \
length_sl=elev_lid792_1m.1mm.ls.sfd slope_st=elev_lid792_1m.1mm.sls.sfd -s

# MFD flow distribution
r.watershed elev=elev_lid792_1m.1mm threshold=3000 \
accum=elev_lid792_1m.1mm.acc.mfd drain=elev_lid792_1m.1mm.drain.mfd \
basin=elev_lid792_1m.1mm.basin.mfd stream=elev_lid792_1m.1mm.stream.mfd \
half_b=elev_lid792_1m.1mm.halfb.mfd vis=elev_lid792_1m.1mm.viz.mfd \
length_sl=elev_lid792_1m.1mm.ls.mfd slope_st=elev_lid792_1m.1mm.sls.mfd \
convergence=5

Testing commands for the elevation map elevation.10m in the Spearfish sample dataset spearfish60. Differences between SFD and MFD are mainly in the northern half.

g.region rast=elevation.10m
# multiply elevation.10m to get centimetres as vertical units
r.mapcalc "elevation.10m.1cm = round(elevation.10m * 100.0)"

# SFD (D8) flow distribution
r.watershed elev=elevation.10m.1cm threshold=3000 \
accum=elevation.10m.1cm.acc.sfd drain=elevation.10m.1cm.drain.sfd \
basin=elevation.10m.1cm.basin.sfd stream=elevation.10m.1cm.stream.sfd \
half_b=elevation.10m.1cm.halfb.sfd vis=elevation.10m.1cm.viz.sfd \
length_sl=elevation.10m.1cm.ls.sfd slope_st=elevation.10m.1cm.sls.sfd -s

# MFD flow distribution
r.watershed elev=elevation.10m.1cm threshold=3000 \
accum=elevation.10m.1cm.acc.mfd drain=elevation.10m.1cm.drain.mfd \
basin=elevation.10m.1cm.basin.mfd stream=elevation.10m.1cm.stream.mfd \
half_b=elevation.10m.1cm.halfb.mfd vis=elevation.10m.1cm.viz.mfd \
length_sl=elevation.10m.1cm.ls.mfd slope_st=elevation.10m.1cm.sls.mfd \
convergence=5

Michael
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Helena Mitasova wrote:

just a quick note from the first run - it may be better to have SFD as default so that the same command as in grass63
gives the same result. E.g. running
g.region rast=elevation
r.watershed elevation thresh=10000 accum=accum_10K2 drain=draindir_10K2 basin=basin_10K2 stream=rivers2

gives an MFD result that some people may consider weird when compared with the old r.watershed
(although it is perfectly fine as MFD result)

This is true, the same command gives a different result. But some other people may consider the SFD result weird, e.g. with elev_lid792_1m in nc_spm_08. As an example imagine a user who wants to compare the output of different modules for flow accumulation and runs r.terraflow and r.watershed with default settings. With DEMs like elev_lid792_1m, MFD is needed to get halfway realistic results, and r.watershed with SFD produces here really weird results. I haven't found yet a testing dataset where I would prefer the SFD results over the MFD results.

So the argument to have SFD as default is that default settings produce results identical to the previous version.
The argument to have MFD as default is to keep default settings similar to r.terraflow and to use the mode as default that is likely to produce the most realistic results (still debatable).

running the same with r.watershed -s gives the same, more "normal" looking result.
So people who have r.watershed in scripts and expect/need SFD result would need to add -s,
or will be puzzled by the different result.

I'm not that much of a hydrology expert, when would SFD results be needed? Should a paragraph be added to the documentation when SFD is preferred over MFD?

So whoever has some examples, scripts with r.watershed please run it to see whether
it behaves as expected.

Helena

Helena Mitasova wrote:

On Jan 17, 2009, at 12:07 AM, Michael Barton wrote:

Markus,

I compiled last night and was able to try this on my Mac this evening. No problem at all with the North Carolina elev_lid792_1m DEM. Everything ran very fast, of course--even though it runs at 32bit on the Mac.

As expected, the MFD results look much more realistic than the SFD results. Much less linearity and more sinuosity. I can post the displays if anyone wants to see them.

No errors from the command.

No errors in compiling.

Seems to work fine on the Mac.

I'm not sure why this can't be backported to develbranch_6 since 6.4RCx is already out. Does it change any of the r.watershed arguments or behavior using the pre-MFD arguments?

Yes, MFD is default, so using the same arguments produces different results. See also my reply to Helena:
http://lists.osgeo.org/pipermail/grass-dev/2009-January/041965.html

I would prefer to stay out of the discussion about backporting and leave this decision to the long-time developers.

Michael
- it needs to be thoroughly tested on wide range of data
before replacing it. I am trying to get to it too and Markus M has done already quite a bit of testing himself. If you can ask Isaac and/or whoever has grass7 and is around to test it with different data
(wide range of resolutions, data sources, combinations of parameters, integer or float DEMs, latlong, different size and type of depressions etc.) that would help.

I fully agree. BTW (repeating myself), the results improve a lot if float DEMs are multiplied with 100 or 1000 before used as input.

Markus M

On Jan 17, 2009, at 5:04 AM, Markus Metz wrote:

Helena Mitasova wrote:

just a quick note from the first run - it may be better to have SFD as default so that the same command as in grass63
gives the same result. E.g. running
g.region rast=elevation
r.watershed elevation thresh=10000 accum=accum_10K2 drain=draindir_10K2 basin=basin_10K2 stream=rivers2

gives an MFD result that some people may consider weird when compared with the old r.watershed
(although it is perfectly fine as MFD result)

This is true, the same command gives a different result. But some other people may consider the SFD result weird, e.g. with elev_lid792_1m in nc_spm_08. As an example imagine a user who wants to compare the output of different modules for flow accumulation and runs r.terraflow and r.watershed with default settings. With DEMs like elev_lid792_1m, MFD is needed to get halfway realistic results, and r.watershed with SFD produces here really weird results. I haven't found yet a testing dataset where I would prefer the SFD results over the MFD results.

I am talking here about the issues related to including the new r.watershed in GRASS64 not the merits
of SFD versus MFD (I have done my share of criticizing SFD for many years)
  - people may have scripts, applications where they expect certain result and that will be very different
with MFD for certain data. This is perfectly fine with GRASS7 but it may be problematic with GRASS64.

Just run it with elevation in nc_spm. The MFD result is actually more realistic because it simulates the existing
lakes but you would have to do additional processing if you need a stream network (or re-run with -s
or to get a really good result switch from MFD to SFD as your modification allows, which is great).

I would like to see the new r.watershed in GRASS64 and not wait for GRASS7 but the change should not
create problems for people who have r.watershed included in their applications or modeling tools.

So the argument to have SFD as default is that default settings produce results identical to the previous version.

that is exactly what I had in mind for GRASS64

The argument to have MFD as default is to keep default settings similar to r.terraflow and to use the mode as default that is likely to produce the most realistic results (still debatable).

MFD does produce more realistic results - but the problem is that is not what people always want
(like the issue of hydrologically "correct" DEMs that fill all the depressions changing your wetlands
into straight channels and creating a totally unrealistic hydrology, but many GIS textbooks tell you
to do just that)

running the same with r.watershed -s gives the same, more "normal" looking result.
So people who have r.watershed in scripts and expect/need SFD result would need to add -s,
or will be puzzled by the different result.

I'm not that much of a hydrology expert, when would SFD results be needed?

when you need to extract vectorized stream network from lower resolution data (30-90m) and you need the network
to create a single channel through lakes, SRTM is a good example as is the elevation DEM in nc_cpm_08.

| Should a paragraph be added to the documentation when SFD is preferred over MFD?

yes, some comments on when to use which option would be helpful.

Helena

So whoever has some examples, scripts with r.watershed please run it to see whether
it behaves as expected.

Helena

Helena Mitasova wrote:

[...]
Just run it with elevation in nc_spm. The MFD result is actually more realistic because it simulates the existing
lakes but you would have to do additional processing if you need a stream network

Have you looked at the stream segments output (option stream)? That is a stream network AFAICT, similar to SFD stream output, and can be vectorized with r.thin followed by r.to.vect, same as the SFD stream output. Using MFD flow accumulation instead of MFD stream segments to get a stream network with different stream segments is however very difficult.

[...]

running the same with r.watershed -s gives the same, more "normal" looking result.
So people who have r.watershed in scripts and expect/need SFD result would need to add -s,
or will be puzzled by the different result.

I'm not that much of a hydrology expert, when would SFD results be needed?

when you need to extract vectorized stream network from lower resolution data (30-90m) and you need the network
to create a single channel through lakes, SRTM is a good example as is the elevation DEM in nc_cpm_08.

See comment above, the stream output does give you a single channel through a lake, also with MFD. I tried to make sure the stream segments output of MFD can be as easily vectorized as the stream segments output of SFD.

| Should a paragraph be added to the documentation when SFD is preferred over MFD?

yes, some comments on when to use which option would be helpful.

Helena

So whoever has some examples, scripts with r.watershed please run it to see whether
it behaves as expected.

Helena

On Jan 17, 2009, at 3:04 AM, Markus Metz wrote:

Helena Mitasova wrote:

just a quick note from the first run - it may be better to have SFD as default so that the same command as in grass63
gives the same result. E.g. running
g.region rast=elevation
r.watershed elevation thresh=10000 accum=accum_10K2 drain=draindir_10K2 basin=basin_10K2 stream=rivers2

gives an MFD result that some people may consider weird when compared with the old r.watershed
(although it is perfectly fine as MFD result)

This is true, the same command gives a different result. But some other people may consider the SFD result weird, e.g. with elev_lid792_1m in nc_spm_08. As an example imagine a user who wants to compare the output of different modules for flow accumulation and runs r.terraflow and r.watershed with default settings. With DEMs like elev_lid792_1m, MFD is needed to get halfway realistic results, and r.watershed with SFD produces here really weird results. I haven't found yet a testing dataset where I would prefer the SFD results over the MFD results.

So the argument to have SFD as default is that default settings produce results identical to the previous version.
The argument to have MFD as default is to keep default settings similar to r.terraflow and to use the mode as default that is likely to produce the most realistic results (still debatable).

running the same with r.watershed -s gives the same, more "normal" looking result.
So people who have r.watershed in scripts and expect/need SFD result would need to add -s,
or will be puzzled by the different result.

I'm not that much of a hydrology expert, when would SFD results be needed? Should a paragraph be added to the documentation when SFD is preferred over MFD?

So whoever has some examples, scripts with r.watershed please run it to see whether
it behaves as expected.

Helena

As I understand it, we should not break backward compatibility within a release or version. That means that r.watershed should have an SFD default for GRASS 6.4 at least and maybe 6.5 (I can't remember if this holds between subversion releases) and MFD could be the default for GRASS 7.

Michael

On Jan 16, 2009, at 11:03 PM, Helena Mitasova wrote:

Helena

P.S. I often do grass stuff from home over the net. For grass7 display through gui, even clicking off or changing
the names of layers is rather time consuming compared to d.rast with x0 (I have cable).
Any hints, tricks on how to display maps fast when testing would be appreciated.

Helena,

I'm not quite sure what you are referring to here, but will try to offer some response. In the wxPython GUI, you can set it to display/render the maps automatically OR turn this off and render the maps manually by clicking the show/display button. This is in preferences. Also, keep it in the default render mode and NOT "Constrain display resolution to computational settings".

I'm sure that things run slower over a network. But on my local machine, the display is much faster in the new GUI than in the old GUI or d.rast via an xmon. I would think it ought to be faster even over the network. The default in the GUI is to render maps for display at screen resolution--often much lower than the map resolution. This makes the display nearly instantaneous locally. It should also result in less info sent over the network when run remotely. However, I don't know how you had xmons configured or the current GUI display configured over a network.

Michael

On 17/01/09 07:03, Helena Mitasova wrote:

P.S. I often do grass stuff from home over the net. For grass7 display through gui, even clicking off or changing
the names of layers is rather time consuming compared to d.rast with x0 (I have cable).
Any hints, tricks on how to display maps fast when testing would be appreciated.

You could try ximgview. See [1] for Glynn's explanations of how it works.

I have also successfully used gqview as my viewer (just point it to map.png and it gets updated automatically).

Not sure how any of this compares performance-wise when used over the net.

Moritz

[1] http://lists.osgeo.org/pipermail/grass-dev/2008-October/040589.html

On Wed, Jan 21, 2009 at 4:53 PM, Moritz Lennert
<mlennert@club.worldonline.be> wrote:

On 17/01/09 07:03, Helena Mitasova wrote:

P.S. I often do grass stuff from home over the net. For grass7 display
through gui, even clicking off or changing
the names of layers is rather time consuming compared to d.rast with x0 (I
have cable).
Any hints, tricks on how to display maps fast when testing would be
appreciated.

You could try ximgview. See [1] for Glynn's explanations of how it works.

I have also successfully used gqview as my viewer (just point it to map.png
and it gets updated automatically).

Could you wrap this up into some d.mon command? Would be cool
unless there is a wx based monitor system.

Markus

Not sure how any of this compares performance-wise when used over the net.

Moritz

[1] http://lists.osgeo.org/pipermail/grass-dev/2008-October/040589.html
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Markus Neteler wrote:

>> P.S. I often do grass stuff from home over the net. For grass7 display
>> through gui, even clicking off or changing
>> the names of layers is rather time consuming compared to d.rast with x0 (I
>> have cable).
>> Any hints, tricks on how to display maps fast when testing would be
>> appreciated.
>
> You could try ximgview. See [1] for Glynn's explanations of how it works.
>
> I have also successfully used gqview as my viewer (just point it to map.png
> and it gets updated automatically).

Could you wrap this up into some d.mon command? Would be cool
unless there is a wx based monitor system.

The main problem with trying to emulate d.mon is that the libraries
use environment variables for most of their configuration parameters,
and one process (e.g. d.mon) can't change the environment of another
process (in this case, the shell).

It has occurred to me that it might be better to use GRASS variables
instead, but that would require changing the GUI to match.

The biggest problem with monitoring the image files generated by the
display drivers is the need to handle the case where the program tries
to read the file while it's being modified.

This isn't an issue for ximgview, as it intentionally only works with
mmap()d 32-bit BMP files. The PNG and cairo drivers can modify these
files in-place, so the file is never in an invalid state.

The polling behaviour of ximgview probably isn't ideal for a remote
connection, as it will continually redraw the display. However, I've
fixed it so that you can specify percent=0, which will prevent the
redrawing.

You can force a redraw by sending SIGUSR1 to the ximgview process, and
you can make this happen whenever a display command completes with
e.g.:

  export GRASS_NOTIFY='kill -USR1 `pidof ximgview`'

A brief guide to using mmap()d BMP files and ximgview (works with both
the PNG and cairo drivers):

  export GRASS_PNGFILE=map.bmp
  export GRASS_PNG_MAPPED=TRUE
  d.erase # create the file
  export GRASS_PNG_READ=TRUE
  ximgview map.bmp percent=0 &
  export GRASS_NOTIFY='kill -USR1 `pidof ximgview`'
  # d.* commands follow

--
Glynn Clements <glynn@gclements.plus.com>

On Wed, Jan 21, 2009 at 11:51 PM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Markus Neteler wrote:

>> P.S. I often do grass stuff from home over the net. For grass7 display
>> through gui, even clicking off or changing
>> the names of layers is rather time consuming compared to d.rast with x0 (I
>> have cable).
>> Any hints, tricks on how to display maps fast when testing would be
>> appreciated.
>
> You could try ximgview. See [1] for Glynn's explanations of how it works.
>
> I have also successfully used gqview as my viewer (just point it to map.png
> and it gets updated automatically).

Could you wrap this up into some d.mon command? Would be cool
unless there is a wx based monitor system.

...

I tried the suggestion had success after changing USR1 to SIGUSR1:

cat d.mon
#!/bin/sh

export GRASS_PNGFILE=/tmp/grass7map.bmp
export GRASS_PNG_MAPPED=TRUE
d.erase # create the file
export GRASS_PNG_READ=TRUE
ximgview map.bmp percent=0 &
export GRASS_NOTIFY='kill -SIGUSR1 `pidof ximgview`'
# d.* commands follow

(I also tried gqview which offers limited zoom capabilities.)

Markus

Markus Neteler wrote:

>> >> P.S. I often do grass stuff from home over the net. For grass7 display
>> >> through gui, even clicking off or changing
>> >> the names of layers is rather time consuming compared to d.rast with x0 (I
>> >> have cable).
>> >> Any hints, tricks on how to display maps fast when testing would be
>> >> appreciated.
>> >
>> > You could try ximgview. See [1] for Glynn's explanations of how it works.
>> >
>> > I have also successfully used gqview as my viewer (just point it to map.png
>> > and it gets updated automatically).
>>
>> Could you wrap this up into some d.mon command? Would be cool
>> unless there is a wx based monitor system.
...

I tried the suggestion had success after changing USR1 to SIGUSR1:

The argument to "kill" should be either a signal number or a signal
name. According to POSIX, signal names shouldn't have the "SIG"
prefix. The GNU version of "kill" and the bash built-in version both
accept signal names with or without the "SIG" prefix.

http://www.opengroup.org/onlinepubs/009695399/utilities/kill.html

cat d.mon
#!/bin/sh

export GRASS_PNGFILE=/tmp/grass7map.bmp
export GRASS_PNG_MAPPED=TRUE
d.erase # create the file
export GRASS_PNG_READ=TRUE
ximgview map.bmp percent=0 &
export GRASS_NOTIFY='kill -SIGUSR1 `pidof ximgview`'
# d.* commands follow

Note that the d.* commands would have to be contained in the script,
as the "export" commands only affect the environment of commands which
are run from the script; they won't affect the environment of the
shell (or other process) from which the script is run.

It isn't possible for one process to change the environment of another
existing process, so you can't write a script which will change the
environment of the shell from which it is run.

However, you could "source" the above script from bash (or any Bourne
shell). That will cause the commands to be executed by the shell
itself, rather than in a child process. That approach can't be used
from e.g. Tcl/Tk or Python, though.

--
Glynn Clements <glynn@gclements.plus.com>