[GRASSLIST:8369] r.terraflow, or system, out of memory

Hi,

i'm running r.terraflow (grass6.0.1) on a large:

nsres: 2
ewres: 2
rows: 10050
cols: 7760

raster and the process seems to stop out of the blue, i know that this module works on the same machine on an subset of the data i'm working on.
Anyway i don't understand if this is an r.terraflow problem (I've searched in the bug/whishlist and didn't find anything related) or a kernel problem, indeed i found in /var/log/messages

kernel: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)

Do someone experienced somethig like that and knows how to fix?

Thanks, Michele.

Do you have enough space on your tmp file?
terraflow use temp file that can result quite big...
it happened to me that was the cause of crashing the command and leaving trash on the tmp dir (i can't remember now but there's a way to define which dir use as temp)....
Don't know if it is related...
Hope can helps.
Maxi

On Wed, 21 Sep 2005 18:16:45 +0200
  "Michele" <michele.rocc@libero.it> wrote:

Hi,

i'm running r.terraflow (grass6.0.1) on a large:

nsres: 2
ewres: 2
rows: 10050
cols: 7760

raster and the process seems to stop out of the blue, i know that this module works on the same machine on an subset of the data i'm working on.
Anyway i don't understand if this is an r.terraflow problem (I've searched in the bug/whishlist and didn't find anything related) or a kernel problem, indeed i found in /var/log/messages

kernel: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)

Do someone experienced somethig like that and knows how to fix?

Thanks, Michele.

_
--------------------------------------
Eng. Massimiliano Cannata
Geomoatic Phd student
www.ist.supsi.ch

It could also be that you have exceeded 2 Gig on one of the temp
files. r.terraflow needs to be compiled with special flags to exceed
this amount...I don't think this is done be default yet,
unfortunately.

Look at the GRASSLIST archives for this message and the follow-ups:
[GRASSLIST:7445]

On 9/21/05, Cannata, Massimiliano <massimiliano.cannata@supsi.ch> wrote:

Do you have enough space on your tmp file?
terraflow use temp file that can result quite big...
it happened to me that was the cause of crashing the
command and leaving trash on the tmp dir (i can't remember
now but there's a way to define which dir use as temp)....
Don't know if it is related...
Hope can helps.
Maxi

On Wed, 21 Sep 2005 18:16:45 +0200
  "Michele" <michele.rocc@libero.it> wrote:
> Hi,
>
> i'm running r.terraflow (grass6.0.1) on a large:
>
> nsres: 2
> ewres: 2
> rows: 10050
> cols: 7760
>
> raster and the process seems to stop out of the blue, i
>know that this module works on the same machine on an
>subset of the data i'm working on.
> Anyway i don't understand if this is an r.terraflow
>problem (I've searched in the bug/whishlist and didn't
>find anything related) or a kernel problem, indeed i
>found in /var/log/messages
>
> kernel: __alloc_pages: 0-order allocation failed
>(gfp=0xf0/0)
>
> Do someone experienced somethig like that and knows how
>to fix?
>
> Thanks, Michele.
>

_
--------------------------------------
Eng. Massimiliano Cannata
Geomoatic Phd student
www.ist.supsi.ch

--
David Finlayson
Marine Geology & Geophysics
School of Oceanography
Box 357940
University of Washington
Seattle, WA 98195-7940
USA

Office: Marine Sciences Building, Room 112
Phone: (206) 616-9407
Web: http://students.washington.edu/dfinlays

> > i'm running r.terraflow (grass6.0.1) on a large:
> >
> > nsres: 2
> > ewres: 2
> > rows: 10050
> > cols: 7760
> >
> > raster and the process seems to stop out of the blue, i
> >know that this module works on the same machine on an
> >subset of the data i'm working on.
> > Anyway i don't understand if this is an r.terraflow
> >problem (I've searched in the bug/whishlist and didn't
> >find anything related) or a kernel problem, indeed i
> >found in /var/log/messages
> >
> > kernel: __alloc_pages: 0-order allocation failed
> >(gfp=0xf0/0)
> >
> > Do someone experienced somethig like that and knows how
> >to fix?

..

> Do you have enough space on your tmp file?
> terraflow use temp file that can result quite big...
> it happened to me that was the cause of crashing the
> command and leaving trash on the tmp dir (i can't remember
> now but there's a way to define which dir use as temp)....
> Don't know if it is related...

..

It could also be that you have exceeded 2 Gig on one of the temp
files. r.terraflow needs to be compiled with special flags to exceed
this amount...I don't think this is done be default yet,
unfortunately.

Look at the GRASSLIST archives for this message and the follow-ups:
[GRASSLIST:7445]

here's that thread:
  http://thread.gmane.org/gmane.comp.gis.grass.user/8868

From the r.terraflow help page:

"One of the techniques used by r.terraflow is the space-time trade-off.
In particular, in order to avoid searches, which are I/O-expensive,
r.terraflow computes and works with an augmented elevation raster in
which each cell stores relevant information about its 8 neighbors, in
total up to 80B per cell. As a result r.terraflow works with
intermediate temporary files that may be up to 80N bytes, where N is the
number of cells (rows x columns) in the elevation raster (more
precisely, 80K bytes, where K is the number of valid (not nodata) cells
in the input elevation raster). All this intermediate temporary files
are stored in the path specified by STREAM_DIR. Note: STREAM_DIR must
contain enough free disk space in order to store up to 2 x 80N bytes."

So 10050 x 7760 cells * 80 bytes * 2 = ~12gb disk space needed, and as
David mentions this is bigger than 2GB so you will need to have GRASS
built with large file support*. If you did build GRASS with large file
support, then this is a bug as the r.terraflow Makefiles aren't checking
for that flag and making the needed adjustments. I'm pretty sure that
this has not been fixed but don't know enough about LFS and the
inner workings of the configure system to fix it myself.

[*] ./configure --with-cxx --enable-largefile

Hamish