[GRASS5] error encountered in r.terraflow

GRASS GIS GRASS GIS developer
,
1

Hello list,

after the prior process of getting 2gb data conversion using gdal with grass61cvs been a success.

i am facing a problem with terraflow process on the 2gb dataset(ESRI Ascii Raster Grid) which i converted to grass raster grid file.

r.terraflow December 2003
region size is 19954 x 21199
cell newdata header compatible with region header
elevation stored as FLOAT (4B) WARNING: raster newdata is of type CELL_TYPE --you should use r.terraflow.short

STREAM temporary files in /var/tmp (THESE INTERMEDIATE STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES MANUALLY!)
SFD (D8) flow direction
D8CUT=999999986991104.000000
memory size: 1000.00M (1048576000) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
reading data from newdata to stream /var/tmp/STREAM_NnrFqS:

total elements=423004846, nodata elements=153409740
largest temporary files:
FILL: 22.06G (23688271376) [423004846 elements, 56B each]
FLOW: 20.09G (21567608480) [269595106 elements, 80B each]
Will need at least 44.12G (47376542752) space available in /var/tmp

COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
finding nodata
relabeling nodata
MM warning: limit=1048576000B. allocating 524140056B. limit exceeded by 58762B.
:
:
merging relabeled grid
MM warning: limit=1048576000B. allocating 524140056B. limit exceeded by 58762B.
:
:

assigning preliminary directions

finding flat areas (plateaus and depressions)
plateau.cc:starting memscan
plateau.cc:memscan done
removing duplicate plateaus
MM warning: limit=1048576000B. allocating 524140060B. limit exceeded by 58770B.
:
relabeling plateaus
MM warning: limit=1048576000B. allocating 524140060B. limit exceeded by 58766B.
generating plateau statistics
MM warning: limit=1048576000B. allocating 523975900B. limit exceeded by 58766B.

assigning directions on plateaus
MM warning: limit=1048576000B. allocating 524222112B. limit exceeded by 48098B.

generating watersheds and watershed graph
creating windows
warning: using slower scan
r.terraflow: 3scan.h:163: void scan3line(FUN&, AMI_STREAM, AMI_STREAM, AMI_STREAM*, BASETYPE, dimension_type) [with T = waterGridType, BASETYPE = waterWindowBaseType, FUN = waterWindower]: Assertion `ae == AMI_ERROR_END_OF_STREAM’ failed.

and also seem not to process data above 1gb(ESRI Ascii Raster Grid) too.
is there anything else i missed while compilation of grass?

PS. i had this was my configuration, compilation and installation statement from grass61cvs.

./configure --enable-shared --with-fftw --with-includes=/usr/include --with-libs=/usr/lib --with-motif --with-freetype=yes --with-freetype-includes=/usr/include/freetype2 --with-nls --with-gdal=/usr/local/bin/gdal-config --with-proj --with-proj-includes=/usr/include --with-proj-libs=/usr/lib --with-glw --with-cxx --with-mysql --with-mysql-includes=/usr/include/mysql --with-mysql-libs=/usr/lib/mysql --with-postgres --with-postgres-includes=/usr/include/pgsql --with-postgres-libs=/usr/lib --with-odbc=yes --with-odbc-includes=/usr/include --with-odbc-libs=/usr/lib --with-glw=yes --enable-largefile;
make;
make install

–
Best Regrads

Abhay Menon

2

My response to the same question, on GRASSLIST:

This error occurs because currently Terraflow uses the standard C file stream calls. The permanent solution to this would be to add some Makefile magic to detect the current platform, and use the appropriate stream types. The temporary solution is to use the patch I've attached. You'll need the grass source tree, to apply the patch do something like:

cd /grass6/raster/r.terraflow/IOStream/include
patch < ami_stream.h.patch
cd ../..
make clean
make

Then install this modified terraflow.

Do any GRASS devs have the time and expertise to make terraflow largefile aware? In my experience this (very minor) patch does the trick, should be just fseek / ftell which need proper detection

Cheers,
Shaun Walbridge

abhay menon wrote:

Hello list,
after the prior process of getting 2gb data conversion using gdal with grass61cvs been a success.
i am facing a problem with terraflow process on the 2gb dataset(ESRI Ascii Raster Grid) which i converted to grass raster grid file.
r.terraflow December 2003
region size is 19954 x 21199
cell newdata header compatible with region header
elevation stored as FLOAT (4B) WARNING: raster newdata is of type CELL_TYPE --you should use r.terraflow.short

STREAM temporary files in /var/tmp (THESE INTERMEDIATE STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES MANUALLY!)
SFD (D8) flow direction
D8CUT=999999986991104.000000
memory size: 1000.00M (1048576000) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
reading data from newdata to stream /var/tmp/STREAM_NnrFqS:

total elements=423004846, nodata elements=153409740
largest temporary files:
   FILL: 22.06G (23688271376) [423004846 elements, 56B each]
   FLOW: 20.09G (21567608480) [269595106 elements, 80B each]
Will need at least 44.12G (47376542752) space available in /var/tmp
------------------------------
COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
finding nodata
relabeling nodata
MM warning: limit=1048576000B. allocating 524140056B. limit exceeded by 58762B.
:
merging relabeled grid
MM warning: limit=1048576000B. allocating 524140056B. limit exceeded by 58762B.
:
----------
assigning preliminary directions
----------
finding flat areas (plateaus and depressions)
plateau.cc:starting memscan
plateau.cc:memscan done
removing duplicate plateaus
MM warning: limit=1048576000B. allocating 524140060B. limit exceeded by 58770B.
:
relabeling plateaus
MM warning: limit=1048576000B. allocating 524140060B. limit exceeded by 58766B.
generating plateau statistics
MM warning: limit=1048576000B. allocating 523975900B. limit exceeded by 58766B.
----------
assigning directions on plateaus
MM warning: limit=1048576000B. allocating 524222112B. limit exceeded by 48098B.
--------------
generating watersheds and watershed graph
creating windows
warning: using slower scan
r.terraflow: 3scan.h:163: void scan3line(FUN&, AMI_STREAM<T>*, AMI_STREAM<T>*, AMI_STREAM<T>*, BASETYPE, dimension_type) [with T = waterGridType, BASETYPE = waterWindowBaseType, FUN = waterWindower]: Assertion `ae == AMI_ERROR_END_OF_STREAM' failed.

and also seem not to process data above 1gb(ESRI Ascii Raster Grid) too.
is there anything else i missed while compilation of grass?
PS. i had this was my configuration, compilation and installation statement from grass61cvs.
./configure --enable-shared --with-fftw --with-includes=/usr/include --with-libs=/usr/lib --with-motif --with-freetype=yes --with-freetype-includes=/usr/include/freetype2 --with-nls --with-gdal=/usr/local/bin/gdal-config --with-proj --with-proj-includes=/usr/include --with-proj-libs=/usr/lib --with-glw --with-cxx --with-mysql --with-mysql-includes=/usr/include/mysql --with-mysql-libs=/usr/lib/mysql --with-postgres --with-postgres-includes=/usr/include/pgsql --with-postgres-libs=/usr/lib --with-odbc=yes --with-odbc-includes=/usr/include --with-odbc-libs=/usr/lib --with-glw=yes --enable-largefile;
make;
make install

--
Best Regrads

Abhay Menon

3

Yes, the problem is related to using fseek and ftell (which are only 32
bit) instead of fseeko/ftello which support off_t offsets which can be
64-bits long if CPPFLAGS includes -D_FILE_OFFSET_BITS=64
-D_LARGEFILE_SOURCE.

I'm not sure if this completely fixes the problem. I would have to see
the patch, but I thought some of the terraflow types for storing offsets
were of type "long" instead of "off_t", so some additional tweaking
needs to be done.

I have some familiarity with Terraflow, but I have been concentrating my
efforts in developing a new Terraflow-like module that should be more
robust and have some additional features. If you have a patch that
works, could you submit it to CVS?

If I get some free time, I can look into fixes if things still do not
work after the patch.

-Andy

On Thu, 2006-04-27 at 13:38 -0700, Shaun Walbridge wrote:

My response to the same question, on GRASSLIST:

This error occurs because currently Terraflow uses the standard C file
stream calls. The permanent solution to this would be to add some
Makefile magic to detect the current platform, and use the appropriate
stream types. The temporary solution is to use the patch I've attached.
You'll need the grass source tree, to apply the patch do something like:

cd /grass6/raster/r.terraflow/IOStream/include
patch < ami_stream.h.patch
cd ../..
make clean
make

Then install this modified terraflow.

Do any GRASS devs have the time and expertise to make terraflow
largefile aware? In my experience this (very minor) patch does the
trick, should be just fseek / ftell which need proper detection

Cheers,
Shaun Walbridge

abhay menon wrote:

> Hello list,
>
> after the prior process of getting 2gb data conversion using gdal with
> grass61cvs been a success.
>
> i am facing a problem with terraflow process on the 2gb dataset(ESRI
> Ascii Raster Grid) which i converted to grass raster grid file.
>
> r.terraflow December 2003
> region size is 19954 x 21199
> cell newdata header compatible with region header
> elevation stored as FLOAT (4B) WARNING: raster newdata is of type
> CELL_TYPE --you should use r.terraflow.short
>
> STREAM temporary files in /var/tmp (THESE INTERMEDIATE STREAMS WILL
> NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE
> SPACE PLEASE DELETE THESE FILES MANUALLY!)
> SFD (D8) flow direction
> D8CUT=999999986991104.000000
> memory size: 1000.00M (1048576000) bytes
> Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
> reading data from newdata to stream /var/tmp/STREAM_NnrFqS:
>
> total elements=423004846, nodata elements=153409740
> largest temporary files:
> FILL: 22.06G (23688271376) [423004846 elements, 56B each]
> FLOW: 20.09G (21567608480) [269595106 elements, 80B each]
> Will need at least 44.12G (47376542752) space available in /var/tmp
> ------------------------------
> COMPUTING FLOW DIRECTIONS
> classifying nodata (inner & boundary)
> finding nodata
> relabeling nodata
> MM warning: limit=1048576000B. allocating 524140056B. limit exceeded
> by 58762B.
> :
> :
> merging relabeled grid
> MM warning: limit=1048576000B. allocating 524140056B. limit exceeded
> by 58762B.
> :
> :
> ----------
> assigning preliminary directions
> ----------
> finding flat areas (plateaus and depressions)
> plateau.cc:starting memscan
> plateau.cc:memscan done
> removing duplicate plateaus
> MM warning: limit=1048576000B. allocating 524140060B. limit exceeded
> by 58770B.
> :
> relabeling plateaus
> MM warning: limit=1048576000B. allocating 524140060B. limit exceeded
> by 58766B.
> generating plateau statistics
> MM warning: limit=1048576000B. allocating 523975900B. limit exceeded
> by 58766B.
> ----------
> assigning directions on plateaus
> MM warning: limit=1048576000B. allocating 524222112B. limit exceeded
> by 48098B.
> --------------
> generating watersheds and watershed graph
> creating windows
> warning: using slower scan
> r.terraflow: 3scan.h:163: void scan3line(FUN&, AMI_STREAM<T>*,
> AMI_STREAM<T>*, AMI_STREAM<T>*, BASETYPE, dimension_type) [with T =
> waterGridType, BASETYPE = waterWindowBaseType, FUN = waterWindower]:
> Assertion `ae == AMI_ERROR_END_OF_STREAM' failed.
>
>
> and also seem not to process data above 1gb(ESRI Ascii Raster Grid) too.
> is there anything else i missed while compilation of grass?
>
> PS. i had this was my configuration, compilation and installation
> statement from grass61cvs.
>
> ./configure --enable-shared --with-fftw --with-includes=/usr/include
> --with-libs=/usr/lib --with-motif --with-freetype=yes
> --with-freetype-includes=/usr/include/freetype2 --with-nls
> --with-gdal=/usr/local/bin/gdal-config --with-proj
> --with-proj-includes=/usr/include --with-proj-libs=/usr/lib --with-glw
> --with-cxx --with-mysql --with-mysql-includes=/usr/include/mysql
> --with-mysql-libs=/usr/lib/mysql --with-postgres
> --with-postgres-includes=/usr/include/pgsql
> --with-postgres-libs=/usr/lib --with-odbc=yes
> --with-odbc-includes=/usr/include --with-odbc-libs=/usr/lib
> --with-glw=yes --enable-largefile;
> make;
> make install
>
> --
> Best Regrads
>
> Abhay Menon

_______________________________________________
grass5 mailing list
grass5@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass5

4

Apologies, I forgot to re-attach the patch to this list. The only changes I've found necessary are those couple spots in ami_stream.h. We've successfully run Terraflow on 4GB+ datasets after the swap to fseeko/ftello.

Cheers,
Shaun

Andrew Danner wrote:

Yes, the problem is related to using fseek and ftell (which are only 32
bit) instead of fseeko/ftello which support off_t offsets which can be
64-bits long if CPPFLAGS includes -D_FILE_OFFSET_BITS=64
-D_LARGEFILE_SOURCE.

I'm not sure if this completely fixes the problem. I would have to see
the patch, but I thought some of the terraflow types for storing offsets
were of type "long" instead of "off_t", so some additional tweaking
needs to be done.

I have some familiarity with Terraflow, but I have been concentrating my
efforts in developing a new Terraflow-like module that should be more
robust and have some additional features. If you have a patch that
works, could you submit it to CVS?

If I get some free time, I can look into fixes if things still do not
work after the patch.

-Andy

On Thu, 2006-04-27 at 13:38 -0700, Shaun Walbridge wrote:

My response to the same question, on GRASSLIST:

This error occurs because currently Terraflow uses the standard C file stream calls. The permanent solution to this would be to add some Makefile magic to detect the current platform, and use the appropriate stream types. The temporary solution is to use the patch I've attached. You'll need the grass source tree, to apply the patch do something like:

cd /grass6/raster/r.terraflow/IOStream/include
patch < ami_stream.h.patch
cd ../..
make clean
make

Then install this modified terraflow.

Do any GRASS devs have the time and expertise to make terraflow largefile aware? In my experience this (very minor) patch does the trick, should be just fseek / ftell which need proper detection

Cheers,
Shaun Walbridge

abhay menon wrote:

Hello list,

after the prior process of getting 2gb data conversion using gdal with grass61cvs been a success.

i am facing a problem with terraflow process on the 2gb dataset(ESRI Ascii Raster Grid) which i converted to grass raster grid file.

r.terraflow December 2003
region size is 19954 x 21199
cell newdata header compatible with region header
elevation stored as FLOAT (4B) WARNING: raster newdata is of type CELL_TYPE --you should use r.terraflow.short

STREAM temporary files in /var/tmp (THESE INTERMEDIATE STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES MANUALLY!)
SFD (D8) flow direction
D8CUT=999999986991104.000000
memory size: 1000.00M (1048576000) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
reading data from newdata to stream /var/tmp/STREAM_NnrFqS:

total elements=423004846, nodata elements=153409740
largest temporary files:
  FILL: 22.06G (23688271376) [423004846 elements, 56B each]
  FLOW: 20.09G (21567608480) [269595106 elements, 80B each]
Will need at least 44.12G (47376542752) space available in /var/tmp
------------------------------
COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
finding nodata
relabeling nodata
MM warning: limit=1048576000B. allocating 524140056B. limit exceeded by 58762B.
:
merging relabeled grid
MM warning: limit=1048576000B. allocating 524140056B. limit exceeded by 58762B.
:
----------
assigning preliminary directions
----------
finding flat areas (plateaus and depressions)
plateau.cc:starting memscan
plateau.cc:memscan done
removing duplicate plateaus
MM warning: limit=1048576000B. allocating 524140060B. limit exceeded by 58770B.
:
relabeling plateaus
MM warning: limit=1048576000B. allocating 524140060B. limit exceeded by 58766B.
generating plateau statistics
MM warning: limit=1048576000B. allocating 523975900B. limit exceeded by 58766B.
----------
assigning directions on plateaus
MM warning: limit=1048576000B. allocating 524222112B. limit exceeded by 48098B.
--------------
generating watersheds and watershed graph
creating windows
warning: using slower scan
r.terraflow: 3scan.h:163: void scan3line(FUN&, AMI_STREAM<T>*, AMI_STREAM<T>*, AMI_STREAM<T>*, BASETYPE, dimension_type) [with T = waterGridType, BASETYPE = waterWindowBaseType, FUN = waterWindower]: Assertion `ae == AMI_ERROR_END_OF_STREAM' failed.

and also seem not to process data above 1gb(ESRI Ascii Raster Grid) too.
is there anything else i missed while compilation of grass?

PS. i had this was my configuration, compilation and installation statement from grass61cvs.

./configure --enable-shared --with-fftw --with-includes=/usr/include --with-libs=/usr/lib --with-motif --with-freetype=yes --with-freetype-includes=/usr/include/freetype2 --with-nls --with-gdal=/usr/local/bin/gdal-config --with-proj --with-proj-includes=/usr/include --with-proj-libs=/usr/lib --with-glw --with-cxx --with-mysql --with-mysql-includes=/usr/include/mysql --with-mysql-libs=/usr/lib/mysql --with-postgres --with-postgres-includes=/usr/include/pgsql --with-postgres-libs=/usr/lib --with-odbc=yes --with-odbc-includes=/usr/include --with-odbc-libs=/usr/lib --with-glw=yes --enable-largefile;
make;
make install

--
Best Regrads

Abhay Menon
     

_______________________________________________
grass5 mailing list
grass5@grass.itc.it
http://grass.itc.it/mailman/listinfo/grass5
   

(attachments)

ami_stream.h.patch (1.48 KB)

5

hello all,

am still facing this problem… even after updating the patch file…

r.terraflow December 2003
region size is 13593 x 15215
cell 960MBdata header compatible with region header
elevation stored as FLOAT (4B) WARNING: raster 960MBdata is of type CELL_TYPE --you should use r.terraflow.short

STREAM temporary files in /var/tmp (THESE INTERMEDIATE STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES MANUALLY!)
SFD (D8) flow direction
D8CUT=999999986991104.000000
memory size: 400.00M (419430400) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
reading data from 960MBdata to stream /var/tmp/STREAM_kohqGU:

total elements=206817495, nodata elements=50042031
largest temporary files:
FILL: 10.79G (11581779720) [206817495 elements, 56B each]
FLOW: 11.68G (12542037120) [156775464 elements, 80B each]
Will need at least 23.36G (25084074240) space available in /var/tmp

COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
finding nodata
relabeling nodata
MM warning: limit=419430400B. allocating 209608296B. limit exceeded by 23602B.
MM warning: limit=419430400B. allocating 209608296B. limit exceeded by 23633B.
merging relabeled grid
MM warning: limit=419430400B. allocating 209567256B. limit exceeded by 23558B.
MM warning: limit=419430400B. allocating 209567256B. limit exceeded by 23589B.

assigning preliminary directions

finding flat areas (plateaus and depressions)
plateau.cc:starting memscan
plateau.cc:memscan done
removing duplicate plateaus
MM warning: limit=419430400B. allocating 209567260B. limit exceeded by 23574B.
:
MM warning: limit=419430400B. allocating 209567260B. limit exceeded by 23636B.
relabeling plateaus
MM warning: limit=419430400B. allocating 209567260B. limit exceeded by 23570B.
MM warning: limit=419430400B. allocating 209567260B. limit exceeded by 23601B.
generating plateau statistics
MM warning: limit=419430400B. allocating 209403100B. limit exceeded by 23570B.
MM warning: limit=419430400B. allocating 209403100B. limit exceeded by 23601B.

assigning directions on plateaus
MM warning: limit=419430400B. allocating 209649312B. limit exceeded by 19306B.
:
MM warning: limit=419430400B. allocating 209649312B. limit exceeded by 19368B.

generating watersheds and watershed graph
creating windows
warning: using slower scan
generateWatersheds
MM warning: limit=419430400B. allocating 209628776B. limit exceeded by 10438B.
:
:
:
:
:
MM warning: limit=419430400B. allocating 209628776B. limit exceeded by 11182B.
file=/var/tmp/STREAM_UpisEN:cannot read!: Input/output error
r.terraflow: ./IOStream/include/ami_sort_impl.h:109: void makeRun_Block(AMI_STREAM, T, unsigned int, Compare*) [with T = waterWindowType, Compare = priorityCmpWaterWindowType]: Assertion `err == AMI_ERROR_NO_ERROR’ failed.

–
Best Regrads

Abhay Menon

On 4/28/06, Shaun Walbridge <walbridge@nceas.ucsb.edu > wrote:

Apologies, I forgot to re-attach the patch to this list. The only
changes I’ve found necessary are those couple spots in ami_stream.h.
We’ve successfully run Terraflow on 4GB+ datasets after the swap to
fseeko/ftello.

Cheers,
Shaun

— ami_stream.h 2004-11-09 05:29:58.000000000 -0800
+++ ami_stream.h.edit 2006-04-27 10:44: 48.000000000 -0700
@@ -394,8 +394,8 @@
seek_offset = offset * sizeof(T);
}

  • if (fseek(fp, seek_offset, SEEK_SET) == -1) {
  • cerr << “AMI_STREAM::seek offset=” << seek_offset << “failed.\n”;
  • if (fseeko(fp, seek_offset, SEEK_SET) == -1) {
  • cerr << “AMI_STREAM::seek offset=” << seek_offset << " failed.\n";
    assert(0);
    exit(1);
    }
    @@ -463,7 +463,7 @@

assert(fp);
//if we go past substream range

  • if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
  • if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
    return AMI_ERROR_END_OF_STREAM;

} else {
@@ -489,7 +489,7 @@
assert(fp);

//if we go past substream range

  • if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
  • if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
    return AMI_ERROR_END_OF_STREAM;

} else {
@@ -512,7 +512,7 @@

assert(fp);
//if we go past substream range

  • if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
  • if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
    return AMI_ERROR_END_OF_STREAM;

} else {
@@ -532,7 +532,7 @@

assert(fp);
//if we go past substream range

  • if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
  • if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
    return AMI_ERROR_END_OF_STREAM;

} else {

6

yes tried that tooo…

On 4/28/06, Hamish <hamish_nospam@yahoo.com> wrote:

elevation stored as FLOAT (4B) WARNING: raster 960MBdata is of type
CELL_TYPE --you should use r.terraflow.short

did you try r.terraflow.short ?

Hamish

–
Best Regrads

Abhay Menon

5C, Pocket -5, MIG Flats,
Mayur Vihar Phase - III,
Delhi - 110 096.
Home# 011-22610865
Fax# 011-22610842
Mobile# 9899041254/9810843921

Office# 0120-251 1102 (EXTN 2517,2518).
0120-251 2101 (EXTN 2517,2518).

7

Do you have an large amount of disk space available on your /var partition? The error:
file=/var/tmp/STREAM_UpisEN:cannot read!: Input/output error
indicates you'd run out of space. As mentioned in the help for r.terraflow, it requires 80N where N is rows x columns of space to compute the output. This can be upwards of 40G for large rasters.

cheers,
Shaun

abhay menon wrote:

hello all,
am still facing this problem.... even after updating the patch file..

r.terraflow December 2003
region size is 13593 x 15215
cell 960MBdata header compatible with region header
elevation stored as FLOAT (4B) WARNING: raster 960MBdata is of type CELL_TYPE --you should use r.terraflow.short
STREAM temporary files in /var/tmp (THESE INTERMEDIATE STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES MANUALLY!)
SFD (D8) flow direction
D8CUT=999999986991104.000000
memory size: 400.00M (419430400) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
reading data from 960MBdata to stream /var/tmp/STREAM_kohqGU:
total elements=206817495, nodata elements=50042031
largest temporary files:
   FILL: 10.79G (11581779720) [206817495 elements, 56B each]
   FLOW: 11.68G (12542037120) [156775464 elements, 80B each]
Will need at least 23.36G (25084074240) space available in /var/tmp
------------------------------
COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
finding nodata
relabeling nodata
MM warning: limit=419430400B. allocating 209608296B. limit exceeded by 23602B.
MM warning: limit=419430400B. allocating 209608296B. limit exceeded by 23633B.
merging relabeled grid
MM warning: limit=419430400B. allocating 209567256B. limit exceeded by 23558B.
MM warning: limit=419430400B. allocating 209567256B. limit exceeded by 23589B.
----------
assigning preliminary directions
----------
finding flat areas (plateaus and depressions)
plateau.cc:starting memscan
plateau.cc:memscan done
removing duplicate plateaus
MM warning: limit=419430400B. allocating 209567260B. limit exceeded by 23574B.
:
MM warning: limit=419430400B. allocating 209567260B. limit exceeded by 23636B.
relabeling plateaus
MM warning: limit=419430400B. allocating 209567260B. limit exceeded by 23570B.
MM warning: limit=419430400B. allocating 209567260B. limit exceeded by 23601B.
generating plateau statistics
MM warning: limit=419430400B. allocating 209403100B. limit exceeded by 23570B.
MM warning: limit=419430400B. allocating 209403100B. limit exceeded by 23601B.
----------
assigning directions on plateaus
MM warning: limit=419430400B. allocating 209649312B. limit exceeded by 19306B.
:
MM warning: limit=419430400B. allocating 209649312B. limit exceeded by 19368B.
--------------
generating watersheds and watershed graph
creating windows
warning: using slower scan
generateWatersheds
MM warning: limit=419430400B. allocating 209628776B. limit exceeded by 10438B.
:
MM warning: limit=419430400B. allocating 209628776B. limit exceeded by 11182B.
file=/var/tmp/STREAM_UpisEN:cannot read!: Input/output error
r.terraflow: ./IOStream/include/ami_sort_impl.h:109: void makeRun_Block(AMI_STREAM<T>*, T*, unsigned int, Compare*) [with T = waterWindowType, Compare = priorityCmpWaterWindowType]: Assertion `err == AMI_ERROR_NO_ERROR' failed.

--
Best Regrads

Abhay Menon

On 4/28/06, *Shaun Walbridge* <walbridge@nceas.ucsb.edu <mailto:walbridge@nceas.ucsb.edu> > wrote:

    Apologies, I forgot to re-attach the patch to this list. The only
    changes I've found necessary are those couple spots in ami_stream.h.
    We've successfully run Terraflow on 4GB+ datasets after the swap to
    fseeko/ftello.

    Cheers,
    Shaun

    --- ami_stream.h 2004-11-09 05:29:58.000000000 -0800
    +++ ami_stream.h.edit 2006-04-27 10:44: 48.000000000 -0700
    @@ -394,8 +394,8 @@
        seek_offset = offset * sizeof(T);
      }

    - if (fseek(fp, seek_offset, SEEK_SET) == -1) {
    - cerr << "AMI_STREAM::seek offset=" << seek_offset <<
    "failed.\n";
    + if (fseeko(fp, seek_offset, SEEK_SET) == -1) {
    + cerr << "AMI_STREAM::seek offset=" << seek_offset << "
    failed.\n";
        assert(0);
        exit(1);
      }
    @@ -463,7 +463,7 @@

      assert(fp);
      //if we go past substream range
    - if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
    + if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
        return AMI_ERROR_END_OF_STREAM;

      } else {
    @@ -489,7 +489,7 @@
      assert(fp);

      //if we go past substream range
    - if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
    + if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
        return AMI_ERROR_END_OF_STREAM;

      } else {
    @@ -512,7 +512,7 @@

      assert(fp);
      //if we go past substream range
    - if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
    + if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
        return AMI_ERROR_END_OF_STREAM;

      } else {
    @@ -532,7 +532,7 @@

      assert(fp);
      //if we go past substream range
    - if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
    + if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
        return AMI_ERROR_END_OF_STREAM;

      } else {

8

Shaun,

Thanks! Can someone patch the CVS tree with Shaun's patch? It is
certainly needed for large file support.

-Andy

On Thu, 2006-04-27 at 17:04 -0700, Shaun Walbridge wrote:

Apologies, I forgot to re-attach the patch to this list. The only
changes I've found necessary are those couple spots in ami_stream.h.
We've successfully run Terraflow on 4GB+ datasets after the swap to
fseeko/ftello.

Cheers,
Shaun

Andrew Danner wrote:

>Yes, the problem is related to using fseek and ftell (which are only 32
>bit) instead of fseeko/ftello which support off_t offsets which can be
>64-bits long if CPPFLAGS includes -D_FILE_OFFSET_BITS=64
>-D_LARGEFILE_SOURCE.
>
>I'm not sure if this completely fixes the problem. I would have to see
>the patch, but I thought some of the terraflow types for storing offsets
>were of type "long" instead of "off_t", so some additional tweaking
>needs to be done.
>
>I have some familiarity with Terraflow, but I have been concentrating my
>efforts in developing a new Terraflow-like module that should be more
>robust and have some additional features. If you have a patch that
>works, could you submit it to CVS?
>
>If I get some free time, I can look into fixes if things still do not
>work after the patch.
>
>-Andy
>
>On Thu, 2006-04-27 at 13:38 -0700, Shaun Walbridge wrote:
>
>
>>My response to the same question, on GRASSLIST:
>>
>>This error occurs because currently Terraflow uses the standard C file
>>stream calls. The permanent solution to this would be to add some
>>Makefile magic to detect the current platform, and use the appropriate
>>stream types. The temporary solution is to use the patch I've attached.
>>You'll need the grass source tree, to apply the patch do something like:
>>
>>cd /grass6/raster/r.terraflow/IOStream/include
>>patch < ami_stream.h.patch
>>cd ../..
>>make clean
>>make
>>
>>Then install this modified terraflow.
>>
>>Do any GRASS devs have the time and expertise to make terraflow
>>largefile aware? In my experience this (very minor) patch does the
>>trick, should be just fseek / ftell which need proper detection
>>
>>Cheers,
>>Shaun Walbridge
>>

plain text document attachment (ami_stream.h.patch)
--- ami_stream.h 2004-11-09 05:29:58.000000000 -0800
+++ ami_stream.h.edit 2006-04-27 10:44:48.000000000 -0700
@@ -394,8 +394,8 @@
     seek_offset = offset * sizeof(T);
   }

- if (fseek(fp, seek_offset, SEEK_SET) == -1) {
- cerr << "AMI_STREAM::seek offset=" << seek_offset << "failed.\n";
+ if (fseeko(fp, seek_offset, SEEK_SET) == -1) {
+ cerr << "AMI_STREAM::seek offset=" << seek_offset << " failed.\n";
     assert(0);
     exit(1);
   }
@@ -463,7 +463,7 @@

   assert(fp);
   //if we go past substream range
- if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
+ if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
     return AMI_ERROR_END_OF_STREAM;
   
   } else {
@@ -489,7 +489,7 @@
   assert(fp);
   
   //if we go past substream range
- if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
+ if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
     return AMI_ERROR_END_OF_STREAM;
     
   } else {
@@ -512,7 +512,7 @@

   assert(fp);
   //if we go past substream range
- if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
+ if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
     return AMI_ERROR_END_OF_STREAM;
   
   } else {
@@ -532,7 +532,7 @@

   assert(fp);
   //if we go past substream range
- if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
+ if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
     return AMI_ERROR_END_OF_STREAM;
     
   } else {

9

Also, even if you have a ton of space, if Terraflow has been crashing a
lot, it may leave a bunch of files in /var/tmp which you need to
manually delete. They should probably all look like STREAM_xxxxxx

If you do not have enough space in /var/tmp, but perhaps some other
directory you can use the STREAM_DIR option in terraflow to specify
another location for temp files. Make sure this directory supports
large files. For example, FAT32 file systems can be mounted under
linux/Unix, but cannot support files larger than 2 or 4 GB.

-Andy

On Fri, 2006-04-28 at 08:01 -0700, Shaun Walbridge wrote:

Do you have an large amount of disk space available on your /var
partition? The error:
file=/var/tmp/STREAM_UpisEN:cannot read!: Input/output error
indicates you'd run out of space. As mentioned in the help for
r.terraflow, it requires 80N where N is rows x columns of space to
compute the output. This can be upwards of 40G for large rasters.

cheers,
Shaun

abhay menon wrote:

> hello all,
>
> am still facing this problem.... even after updating the patch file..
>
> r.terraflow December 2003
> region size is 13593 x 15215
> cell 960MBdata header compatible with region header
> elevation stored as FLOAT (4B) WARNING: raster 960MBdata is of type
> CELL_TYPE --you should use r.terraflow.short
> STREAM temporary files in /var/tmp (THESE INTERMEDIATE STREAMS WILL
> NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE
> SPACE PLEASE DELETE THESE FILES MANUALLY!)
> SFD (D8) flow direction
> D8CUT=999999986991104.000000
> memory size: 400.00M (419430400) bytes
> Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
> reading data from 960MBdata to stream /var/tmp/STREAM_kohqGU:
> total elements=206817495, nodata elements=50042031
> largest temporary files:
> FILL: 10.79G (11581779720) [206817495 elements, 56B each]
> FLOW: 11.68G (12542037120) [156775464 elements, 80B each]
> Will need at least 23.36G (25084074240) space available in /var/tmp
> ------------------------------
> COMPUTING FLOW DIRECTIONS
> classifying nodata (inner & boundary)
> finding nodata
> relabeling nodata
> MM warning: limit=419430400B. allocating 209608296B. limit exceeded
> by 23602B.
> MM warning: limit=419430400B. allocating 209608296B. limit exceeded
> by 23633B.
> merging relabeled grid
> MM warning: limit=419430400B. allocating 209567256B. limit exceeded
> by 23558B.
> MM warning: limit=419430400B. allocating 209567256B. limit exceeded
> by 23589B.
> ----------
> assigning preliminary directions
> ----------
> finding flat areas (plateaus and depressions)
> plateau.cc:starting memscan
> plateau.cc:memscan done
> removing duplicate plateaus
> MM warning: limit=419430400B. allocating 209567260B. limit exceeded
> by 23574B.
> :
> MM warning: limit=419430400B. allocating 209567260B. limit exceeded
> by 23636B.
> relabeling plateaus
> MM warning: limit=419430400B. allocating 209567260B. limit exceeded
> by 23570B.
> MM warning: limit=419430400B. allocating 209567260B. limit exceeded
> by 23601B.
> generating plateau statistics
> MM warning: limit=419430400B. allocating 209403100B. limit exceeded
> by 23570B.
> MM warning: limit=419430400B. allocating 209403100B. limit exceeded
> by 23601B.
> ----------
> assigning directions on plateaus
> MM warning: limit=419430400B. allocating 209649312B. limit exceeded
> by 19306B.
> :
> MM warning: limit=419430400B. allocating 209649312B. limit exceeded
> by 19368B.
> --------------
> generating watersheds and watershed graph
> creating windows
> warning: using slower scan
> generateWatersheds
> MM warning: limit=419430400B. allocating 209628776B. limit exceeded
> by 10438B.
> :
> :
> :
> :
> :
> MM warning: limit=419430400B. allocating 209628776B. limit exceeded
> by 11182B.
> file=/var/tmp/STREAM_UpisEN:cannot read!: Input/output error
> r.terraflow: ./IOStream/include/ami_sort_impl.h:109: void
> makeRun_Block(AMI_STREAM<T>*, T*, unsigned int, Compare*) [with T =
> waterWindowType, Compare = priorityCmpWaterWindowType]: Assertion `err
> == AMI_ERROR_NO_ERROR' failed.
>
> --
> Best Regrads
>
> Abhay Menon
>
> On 4/28/06, *Shaun Walbridge* <walbridge@nceas.ucsb.edu
> <mailto:walbridge@nceas.ucsb.edu> > wrote:
>
> Apologies, I forgot to re-attach the patch to this list. The only
> changes I've found necessary are those couple spots in ami_stream.h.
> We've successfully run Terraflow on 4GB+ datasets after the swap to
> fseeko/ftello.
>
> Cheers,
> Shaun
>
> --- ami_stream.h 2004-11-09 05:29:58.000000000 -0800
> +++ ami_stream.h.edit 2006-04-27 10:44: 48.000000000 -0700
> @@ -394,8 +394,8 @@
> seek_offset = offset * sizeof(T);
> }
>
> - if (fseek(fp, seek_offset, SEEK_SET) == -1) {
> - cerr << "AMI_STREAM::seek offset=" << seek_offset <<
> "failed.\n";
> + if (fseeko(fp, seek_offset, SEEK_SET) == -1) {
> + cerr << "AMI_STREAM::seek offset=" << seek_offset << "
> failed.\n";
> assert(0);
> exit(1);
> }
> @@ -463,7 +463,7 @@
>
> assert(fp);
> //if we go past substream range
> - if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
> + if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
> return AMI_ERROR_END_OF_STREAM;
>
> } else {
> @@ -489,7 +489,7 @@
> assert(fp);
>
> //if we go past substream range
> - if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
> + if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
> return AMI_ERROR_END_OF_STREAM;
>
> } else {
> @@ -512,7 +512,7 @@
>
> assert(fp);
> //if we go past substream range
> - if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
> + if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
> return AMI_ERROR_END_OF_STREAM;
>
> } else {
> @@ -532,7 +532,7 @@
>
> assert(fp);
> //if we go past substream range
> - if ((logical_eos >= 0) && ftell(fp) >= sizeof(T) * logical_eos) {
> + if ((logical_eos >= 0) && ftello(fp) >= sizeof(T) * logical_eos) {
> return AMI_ERROR_END_OF_STREAM;
>
> } else {
>

10

Hello Shaun, Andrew and the list,

To Shaun:
yes I do have space…on the system there is about 44.5gb of free space where as the ESRI Ascii Raster grid of size 960mb required 22.62gb as said by the program.

r.terraflow, it requires 80N where N is rows x columns of space to
compute the output. This can be upwards of 40G for large rasters
Is that the Grass Ascii Raster Grid that you are talking about??.

To Andrew:
yes i do manually delete those files in /var/tmp each time when r.terraflow fails.( already warned about the same each time terraflow runs ).

Shaun’s patch did successed in removing the problem with ami_stream.h for the purpose of AMI_STREAM.

but i want to know from Shuan which version of the cvs build did you use, on which your patch work and where able to get the desired results on larger files…

–
Best Regrads

Abhay Menon

On 4/28/06, Shaun Walbridge <walbridge@nceas.ucsb.edu> wrote:

Do you have an large amount of disk space available on your /var
partition? The error:
file=/var/tmp/STREAM_UpisEN:cannot read!: Input/output error
indicates you’d run out of space. As mentioned in the help for
r.terraflow, it requires 80N where N is rows x columns of space to
compute the output. This can be upwards of 40G for large rasters.

cheers,
Shaun

abhay menon wrote:

hello all,

am still facing this problem… even after updating the patch file…

r.terraflow December 2003
region size is 13593 x 15215
cell 960MBdata header compatible with region header
elevation stored as FLOAT (4B) WARNING: raster 960MBdata is of type
CELL_TYPE --you should use r.terraflow.short
STREAM temporary files in /var/tmp (THESE INTERMEDIATE STREAMS WILL
NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE
SPACE PLEASE DELETE THESE FILES MANUALLY!)
SFD (D8) flow direction
D8CUT=999999986991104.000000
memory size: 400.00M (419430400) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
reading data from 960MBdata to stream /var/tmp/STREAM_kohqGU:
total elements=206817495, nodata elements=50042031
largest temporary files:
FILL: 10.79G (11581779720) [206817495 elements, 56B each]
FLOW: 11.68G (12542037120) [156775464 elements, 80B each]
Will need at least 23.36G (25084074240) space available in /var/tmp

COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
finding nodata
relabeling nodata
MM warning: limit=419430400B. allocating 209608296B. limit exceeded
by 23602B.
MM warning: limit=419430400B. allocating 209608296B. limit exceeded
by 23633B.
merging relabeled grid
MM warning: limit=419430400B. allocating 209567256B. limit exceeded
by 23558B.
MM warning: limit=419430400B. allocating 209567256B. limit exceeded
by 23589B.

assigning preliminary directions

finding flat areas (plateaus and depressions)
plateau.cc:starting memscan
plateau.cc:memscan done
removing duplicate plateaus
MM warning: limit=419430400B. allocating 209567260B. limit exceeded
by 23574B.
:
MM warning: limit=419430400B. allocating 209567260B. limit exceeded
by 23636B.
relabeling plateaus
MM warning: limit=419430400B. allocating 209567260B. limit exceeded
by 23570B.
MM warning: limit=419430400B. allocating 209567260B. limit exceeded
by 23601B.
generating plateau statistics
MM warning: limit=419430400B. allocating 209403100B. limit exceeded
by 23570B.
MM warning: limit=419430400B. allocating 209403100B. limit exceeded
by 23601B.

assigning directions on plateaus
MM warning: limit=419430400B. allocating 209649312B. limit exceeded
by 19306B.
:
MM warning: limit=419430400B. allocating 209649312B. limit exceeded
by 19368B.

generating watersheds and watershed graph
creating windows
warning: using slower scan
generateWatersheds
MM warning: limit=419430400B. allocating 209628776B. limit exceeded
by 10438B.
:
:
:
:
:
MM warning: limit=419430400B. allocating 209628776B. limit exceeded
by 11182B.
file=/var/tmp/STREAM_UpisEN:cannot read!: Input/output error
r.terraflow: ./IOStream/include/ami_sort_impl.h:109: void
makeRun_Block(AMI_STREAM, T, unsigned int, Compare*) [with T =
waterWindowType, Compare = priorityCmpWaterWindowType]: Assertion `err
== AMI_ERROR_NO_ERROR’ failed.

–
Best Regrads

Abhay Menon

11

On Fri, 2006-04-28 at 11:06 -0400, Andrew Danner wrote:

Shaun,

Thanks! Can someone patch the CVS tree with Shaun's patch? It is
certainly needed for large file support.

I'll try to get it committed this weekend. I've been doing some work on
the module anyway. I'll see if I can isolate some of the issues
mentioned by Abhay.

--
Brad Douglas <rez touchofmadness com> KB8UYR
Address: 37.493,-121.924 / WGS84 National Map Corps #TNMC-3785

12

hi,

You could test temp space fs with command
dd if=/dev/zero of=/tmp/foo count=83886080
it will create 40Gb zero filled file (count=x*512bytes). If You have
some fs related limits, they will show up.

Maris.

On 4/28/06, abhay menon <abhay.menon@gmail.com> wrote:

Hello Shaun, Andrew and the list,

To Shaun:
yes I do have space...on the system there is about 44.5gb of free space
where as the *ESRI Ascii Raster grid* of size 960mb required 22.62gb as said
by the program.

>r.terraflow, it requires 80N where N is rows x columns of space to
>compute the output. This can be upwards of 40G for large rasters
Is that the Grass Ascii Raster Grid that you are talking about??.

To Andrew:
yes i do manually delete those files in /var/tmp each time when
r.terraflowfails.( already warned about the same each time terraflow
runs ).

Shaun's patch did successed in removing the problem with ami_stream.h for
the purpose of AMI_STREAM.

but i want to know from Shuan which version of the cvs build did you use, on
which your patch work and where able to get the desired results on larger
files..

--
Best Regrads

Abhay Menon

On 4/28/06, Shaun Walbridge <walbridge@nceas.ucsb.edu> wrote:
>
> Do you have an large amount of disk space available on your /var
> partition? The error:
> file=/var/tmp/STREAM_UpisEN:cannot read!: Input/output error
> indicates you'd run out of space. As mentioned in the help for
> r.terraflow, it requires 80N where N is rows x columns of space to
> compute the output. This can be upwards of 40G for large rasters.
>
> cheers,
> Shaun
>
> abhay menon wrote:
>
> > hello all,
> >
> > am still facing this problem.... even after updating the patch file..
> >
> > r.terraflow December 2003
> > region size is 13593 x 15215
> > cell 960MBdata header compatible with region header
> > elevation stored as FLOAT (4B) WARNING: raster 960MBdata is of type
> > CELL_TYPE --you should use r.terraflow.short
> > STREAM temporary files in /var/tmp (THESE INTERMEDIATE STREAMS WILL
> > NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE
> > SPACE PLEASE DELETE THESE FILES MANUALLY!)
> > SFD (D8) flow direction
> > D8CUT=999999986991104.000000
> > memory size: 400.00M (419430400) bytes
> > Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
> > reading data from 960MBdata to stream /var/tmp/STREAM_kohqGU:
> > total elements=206817495, nodata elements=50042031
> > largest temporary files:
> > FILL: 10.79G (11581779720) [206817495 elements, 56B each]
> > FLOW: 11.68G (12542037120) [156775464 elements, 80B each]
> > Will need at least 23.36G (25084074240) space available in /var/tmp
> > ------------------------------
> > COMPUTING FLOW DIRECTIONS
> > classifying nodata (inner & boundary)
> > finding nodata
> > relabeling nodata
> > MM warning: limit=419430400B. allocating 209608296B. limit exceeded
> > by 23602B.
> > MM warning: limit=419430400B. allocating 209608296B. limit exceeded
> > by 23633B.
> > merging relabeled grid
> > MM warning: limit=419430400B. allocating 209567256B. limit exceeded
> > by 23558B.
> > MM warning: limit=419430400B. allocating 209567256B. limit exceeded
> > by 23589B.
> > ----------
> > assigning preliminary directions
> > ----------
> > finding flat areas (plateaus and depressions)
> > plateau.cc:starting memscan
> > plateau.cc:memscan done
> > removing duplicate plateaus
> > MM warning: limit=419430400B. allocating 209567260B. limit exceeded
> > by 23574B.
> > :
> > MM warning: limit=419430400B. allocating 209567260B. limit exceeded
> > by 23636B.
> > relabeling plateaus
> > MM warning: limit=419430400B. allocating 209567260B. limit exceeded
> > by 23570B.
> > MM warning: limit=419430400B. allocating 209567260B. limit exceeded
> > by 23601B.
> > generating plateau statistics
> > MM warning: limit=419430400B. allocating 209403100B. limit exceeded
> > by 23570B.
> > MM warning: limit=419430400B. allocating 209403100B. limit exceeded
> > by 23601B.
> > ----------
> > assigning directions on plateaus
> > MM warning: limit=419430400B. allocating 209649312B. limit exceeded
> > by 19306B.
> > :
> > MM warning: limit=419430400B. allocating 209649312B. limit exceeded
> > by 19368B.
> > --------------
> > generating watersheds and watershed graph
> > creating windows
> > warning: using slower scan
> > generateWatersheds
> > MM warning: limit=419430400B. allocating 209628776B. limit exceeded
> > by 10438B.
> > :
> > MM warning: limit=419430400B. allocating 209628776B. limit exceeded
> > by 11182B.
> > file=/var/tmp/STREAM_UpisEN:cannot read!: Input/output error
> > r.terraflow: ./IOStream/include/ami_sort_impl.h:109: void
> > makeRun_Block(AMI_STREAM<T>*, T*, unsigned int, Compare*) [with T =
> > waterWindowType, Compare = priorityCmpWaterWindowType]: Assertion `err
> > == AMI_ERROR_NO_ERROR' failed.
> >
> > --
> > Best Regrads
> >
> > Abhay Menon
> >