Large-data analysis in GRASS

For the past couple of weeks I have been teaching
myself GRASS. I have gone through a couple of the
tutorials and have become quite fond of the command
syntax (I've a couple of years of ARC/INFO under my
belt). I am earning my masters by doing a large-scale
model of potential bedrock incision in the Himalayas.
Even at 1km resolution, the data set is pretty huge.

How well does GRASS handle very large data sets? On
my home Linux machine (2G free disk space, 64M of RAM)
I have been unable to perform basic watershed analysis
on even a 23M DEM. I was forced to tile the data or
re-sample at a lower resolution. (r.flow,
r.watershed, r.fill.dir tested so far) I would like
to be able to work on data sets many times this size
on my machines at the University. Does anyone have
some words of wisdom on this type of work in GRASS?

I have hand edited most of the major watersheds of the
Himalayas so that GTOPO30 drainage patterns reflect
mapped river courses and collected stream gauging data
for about 30 stations though out the region to help
calibrate the river runoff I am modeling. In the
future, I will be working on drainage extraction
algorithms and I would like to be working to forward
GRASS rather than ESRI. Unfortunately, the canned
routines for drainage extraction (while pretty
interesting) seem to be developed for much smaller
drainage basins (and much smaller data sets).

My immediate need is to somehow replicate the
flowaccumulation command of arc/info so that I can
start accounting for total discharge in my rivers. I
thought that r.watershed would do the trick, but it
couldn't handle even 1 of my basins. Next, it was
suggested that I try r.flow or r.flowmd. I couldn't
find the documentation for r.flowmd so I tried the
former, this would only work on subsets of my data(-M
switch is causing an error) and it is not clear to me
that the streamline density raster is computationally
equivalent to a flowaccumulation grid in arc/info (is
all the water being routed all the way to the
outlets??)

Sorry so long, but I would love to hear how users are
getting around the memory barriers I am finding (or
tell me which book to get my nose into!)

=====
--
David Finlayson
david_finlayson@yahoo.com
University of Washington
Box 351310
Seattle, WA 98195 - 1310

__________________________________________________
Do You Yahoo!?
Yahoo! Photos - 35mm Quality Prints, Now Get 15 Free!
http://photos.yahoo.com/

You might be interested in r.wrat, r.water.fea, d.rast.arrow, SWAT, and
r.mapcalc. As far as that bug, put in a bug report iff there isn't one
already. And as far as memory usage goes, it really varies by program.
Some handle large data sets well, others just puke and die. IMO, it's a
bug for any module to die because of memory iff it doesn't warn you of
this possibility in the man page.

Cheers.
--
/bin/sh ~/.signature:
Command not found

On Thu, 28 Sep 2000, David Finlayson wrote:

How well does GRASS handle very large data sets?

  The SWAT/GRASS combination was used to model the hydrology of the entire
lower 48 states of the US. IIRC, the country was partitioned into > 500
drainage basins and the number crunching took about two weeks on a new,
high-power Sun spark box.

  Bruce Byars at Baylor and Jeff Arnold at the Grasslands Ag Research
Station in Temple, TX can tell you more about it. I don't remember all the
details. But, the important point is that the amount of input data would
qualify under anyone's definition of "very large" and the number crunching
was serious.

  SWAT (Surface Water Assessment Tool) is a comprehensive basin hydrology
model that includes erosion and chemical/sediment dispersion as well as
water runoff. GRASS is used to prepare input data and display output. From
what you wrote, it's the add-on you need. See:
<http://dino.wiz.uni-kassel.de/model_db/mdb/swat.html&gt; for a summary.

Rich

Dr. Richard B. Shepard, President

                       Applied Ecosystem Services, Inc. (TM)
              Making environmentally-responsible mining happen. (SM)
                       --------------------------------
            2404 SW 22nd Street | Troutdale, OR 97060-1247 | U.S.A.
+ 1 503-667-4517 (voice) | + 1 503-667-8863 (fax) | rshepard@appl-ecosys.com