Hi All,
I am trying to process 2GB Ascii raster in Terraflow module with GRASS
version 6.1.cvs.
But it showing following error.
"r.terraflow elev=newdata filled=fill direction=fdir swatershed=sws
accumulation=facc tci=tci d8cut=infinity memory=300 STREAM_DIR=/var/tmp
stats=stats.out -s
r.terraflow December 2003
region size is 19954 x 21199
cell newdata header compatible with region header
elevation stored as FLOAT (4B) WARNING: raster newdata is of type CELL_TYPE
--you should use r.terraflow.short
STREAM temporary files in /var/tmp (THESE INTERMEDIATE STREAMS WILL NOT BE
DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE
PLEASE DELETE THESE FILES MANUALLY!)
SFD (D8) flow direction
D8CUT= 999999986991104.000000
memory size: 300.00M (314572800) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
reading data from newdata to stream /var/tmp/STREAM_n3rZJr:
total elements=423004846, nodata elements=153409740
largest temporary files:
FILL: 22.06G (23688271376) [423004846 elements, 56B each]
FLOW: 20.09G (21567608480) [269595106 elements, 80B each]
Will need at least 44.12G (47376542752) space available in /var/tmp
------------------------------
COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
finding nodata
relabeling nodata
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17734B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17765B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17796B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17827B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17858B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17889B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17920B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17951B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17982B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
18013B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
18044B.
MM warning: limit=314572800B. allocating 157117944B. limit exceeded by
24102B.
merging relabeled grid
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17734B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17765B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17796B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17827B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17858B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17889B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17920B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17951B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17982B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
18013B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
18044B.
----------
assigning preliminary directions
----------
finding flat areas (plateaus and depressions)
plateau.cc:starting memscan
plateau.cc:memscan done
removing duplicate plateaus
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17726B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17757B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17788B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17819B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17850B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17881B.
relabeling plateaus
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17718B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17749B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17780B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17811B.
generating plateau statistics
MM warning: limit=314572800B. allocating 156974296B. limit exceeded by
17718B.
MM warning: limit=314572800B. allocating 156974296B. limit exceeded by
17749B.
MM warning: limit=314572800B. allocating 156974296B. limit exceeded by
17780B.
MM warning: limit=314572800B. allocating 156974296B. limit exceeded by
17811B.
----------
assigning directions on plateaus
MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
14518B.
MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
14549B.
MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
14580B.
MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
14611B.
MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
14642B.
MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
14673B.
--------------
generating watersheds and watershed graph
creating windows
warning: using slower scan
r.terraflow: 3scan.h:163: void scan3line(FUN&, AMI_STREAM<T>*,
AMI_STREAM<T>*, AMI_STREAM<T>*, BASETYPE, dimension_type) [with T =
waterGridType, BASETYPE = waterWindowBaseType, FUN = waterWindower]:
Assertion `ae == AMI_ERROR_END_OF_STREAM' failed.
Any one can help?
Regards
Nagaraj
Hi Nagaraj,
This error occurs because currently Terraflow uses the standard C file stream calls. The permanent solution to this would be to add some Makefile magic to detect the current platform, and use the appropriate stream types. The temporary solution is to use the patch I've attached. You'll need the grass source tree, to apply the patch do something like:
cd /grass6/raster/r.terraflow/IOStream/include
patch < ami_stream.h.patch
cd ../..
make clean
make
Then install this modified terraflow.
Do any GRASS devs have the time and expertise to make terraflow largefile aware? In my experience this (very minor) patch does the trick, should be just fseek / ftell which need proper detection
Cheers,
Shaun Walbridge
Nagaraj Thanushkodi wrote:
Hi All,
I am trying to process 2GB Ascii raster in Terraflow module with GRASS
version 6.1.cvs.
But it showing following error.
"r.terraflow elev=newdata filled=fill direction=fdir swatershed=sws
accumulation=facc tci=tci d8cut=infinity memory=300 STREAM_DIR=/var/tmp
stats=stats.out -s
r.terraflow December 2003
region size is 19954 x 21199
cell newdata header compatible with region header
elevation stored as FLOAT (4B) WARNING: raster newdata is of type CELL_TYPE
--you should use r.terraflow.short
STREAM temporary files in /var/tmp (THESE INTERMEDIATE STREAMS WILL NOT BE
DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE
PLEASE DELETE THESE FILES MANUALLY!)
SFD (D8) flow direction
D8CUT= 999999986991104.000000
memory size: 300.00M (314572800) bytes
Memory manager registering memory in MM_WARN_ON_MEMORY_EXCEEDED mode.
reading data from newdata to stream /var/tmp/STREAM_n3rZJr:
total elements=423004846, nodata elements=153409740
largest temporary files:
FILL: 22.06G (23688271376) [423004846 elements, 56B each]
FLOW: 20.09G (21567608480) [269595106 elements, 80B each]
Will need at least 44.12G (47376542752) space available in /var/tmp
------------------------------
COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
finding nodata
relabeling nodata
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17734B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17765B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17796B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17827B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17858B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17889B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17920B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17951B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
17982B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
18013B.
MM warning: limit=314572800B. allocating 157179492B. limit exceeded by
18044B.
MM warning: limit=314572800B. allocating 157117944B. limit exceeded by
24102B.
merging relabeled grid
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17734B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17765B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17796B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17827B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17858B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17889B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17920B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17951B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
17982B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
18013B.
MM warning: limit=314572800B. allocating 157138452B. limit exceeded by
18044B.
----------
assigning preliminary directions
----------
finding flat areas (plateaus and depressions)
plateau.cc:starting memscan
plateau.cc:memscan done
removing duplicate plateaus
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17726B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17757B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17788B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17819B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17850B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17881B.
relabeling plateaus
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17718B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17749B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17780B.
MM warning: limit=314572800B. allocating 157138456B. limit exceeded by
17811B.
generating plateau statistics
MM warning: limit=314572800B. allocating 156974296B. limit exceeded by
17718B.
MM warning: limit=314572800B. allocating 156974296B. limit exceeded by
17749B.
MM warning: limit=314572800B. allocating 156974296B. limit exceeded by
17780B.
MM warning: limit=314572800B. allocating 156974296B. limit exceeded by
17811B.
----------
assigning directions on plateaus
MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
14518B.
MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
14549B.
MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
14580B.
MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
14611B.
MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
14642B.
MM warning: limit=314572800B. allocating 157220512B. limit exceeded by
14673B.
--------------
generating watersheds and watershed graph
creating windows
warning: using slower scan
r.terraflow: 3scan.h:163: void scan3line(FUN&, AMI_STREAM<T>*,
AMI_STREAM<T>*, AMI_STREAM<T>*, BASETYPE, dimension_type) [with T =
waterGridType, BASETYPE = waterWindowBaseType, FUN = waterWindower]:
Assertion `ae == AMI_ERROR_END_OF_STREAM' failed.
Any one can help?
Regards
Nagaraj
(attachments)
ami_stream.h.patch (1.48 KB)