[GRASS-user] r.terraflow

Francesco_Mirabella · November 9, 2009, 12:24pm

Hi all,
I am trying to get flow directions out of a dem (10m resolution). I have tried r.terraflow which gives me the error below:
Can anyone tell me if I am doing something wrong and how can I solve this?
many thanks
Francesco
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
GRASS 6.4.0RC5 (WGS84_UTM33):~ > r.terraflow elevation=copia.dem filled=flood direction=flow swatershed=sink accumulation=accumulation tci=tci d8cut=infinity memory=300 STREAM_DIR=/tmp stats=stats.out

STREAM temporary files in /tmp (THESE INTERMEDIATE STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES MANUALLY!)
file stats.out exists - renaming.
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 300.00M (314572800) bytes
Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
total elements=67071935, nodata elements=8624611
largest temporary files:
FILL: 3.50G (3756028360) [67071935 elements, 56B each]
FLOW: 4.35G (4675785920) [58447324 elements, 80B each]
Will need at least 8.71G (9351571840) space available in /tmp
------------------------------
COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
EMPQUEUEADAPTIVE: starting in-memory pqueue
EMPQUEUEADAPTIVE: available memory: 297.929MB
EMPQUEUEADAPTIVE: desired memory: 297.929MB
sz_stream: 270388 buf_arity: 200 mm_overhead: 8665728 mm_avail: 312400802.
EMPQUEUEADAPTIVE: memory overhead set to 8.26428MB
EMPQUEUEADAPTIVE: pqsize set to 37966884
assigning preliminary directions
finding flat areas (plateaus and depressions)
file=/tmp/STREAM_rSqNkF:cannot read!: Bad address
r.terraflow: /usr/local/svn/grass/grass640_rc5/dist.i686-pc-linux-gnu/include/grass/iostream/ami_sort_impl.h:91: size_t makeRun_Block(AMI_STREAM<T>*, T*, unsigned int, Compare*) [with T = plateauType, Compare = ijCmpPlateauType]: Assertion `err == AMI_ERROR_NO_ERROR || err == AMI_ERROR_END_OF_STREAM' failed.
Abortito
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

neteler · November 9, 2009, 12:58pm

On Mon, Nov 9, 2009 at 1:24 PM, Francesco Mirabella <mirabell@unipg.it> wrote:

Hi all,
I am trying to get flow directions out of a dem (10m resolution). I have
tried r.terraflow which gives me the error below:
Can anyone tell me if I am doing something wrong and how can I solve this?
many thanks
Francesco
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
GRASS 6.4.0RC5 (WGS84_UTM33):~ > r.terraflow elevation=copia.dem
filled=flood direction=flow swatershed=sink accumulation=accumulation
tci=tci d8cut=infinity memory=300 STREAM_DIR=/tmp stats=stats.out

STREAM temporary files in /tmp (THESE INTERMEDIATE STREAMS WILL NOT BE
DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE
DELETE THESE FILES MANUALLY!)
file stats.out exists - renaming.
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 300.00M (314572800) bytes
Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
total elements=67071935, nodata elements=8624611
largest temporary files:
FILL: 3.50G (3756028360) [67071935 elements, 56B each]
FLOW: 4.35G (4675785920) [58447324 elements, 80B each]
Will need at least 8.71G (9351571840) space available in /tmp
------------------------------
COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
EMPQUEUEADAPTIVE: starting in-memory pqueue
EMPQUEUEADAPTIVE: available memory: 297.929MB
EMPQUEUEADAPTIVE: desired memory: 297.929MB
sz_stream: 270388 buf_arity: 200 mm_overhead: 8665728 mm_avail: 312400802.
EMPQUEUEADAPTIVE: memory overhead set to 8.26428MB
EMPQUEUEADAPTIVE: pqsize set to 37966884
assigning preliminary directions
finding flat areas (plateaus and depressions)
file=/tmp/STREAM_rSqNkF:cannot read!: Bad address
r.terraflow:
/usr/local/svn/grass/grass640_rc5/dist.i686-pc-linux-gnu/include/grass/iostream/ami_sort_impl.h:91:
size_t makeRun_Block(AMI_STREAM<T>*, T*, unsigned int, Compare*) [with T =
plateauType, Compare = ijCmpPlateauType]: Assertion `err ==
AMI_ERROR_NO_ERROR || err == AMI_ERROR_END_OF_STREAM' failed.
Abortito

This is a known bug:
http://trac.osgeo.org/grass/ticket/775

with a patch (r.terraflow.diff) to test. Perhaps you could try it
and report back directly in the ticket?

thanks
Markus

Hamish · November 9, 2009, 1:20pm

Francesco wrote:
----------------------------
GRASS 6.4.0RC5 (WGS84_UTM33):~ > r.terraflow elevation=copia.dem
> filled=flood direction=flow swatershed=sink
accumulation=accumulation
tci=tci d8cut=infinity memory=300 STREAM_DIR=/tmp stats=stats.out

can you set memory= to something higher? (say 80% of physical RAM)

Memory size: 300.00M (314572800) bytes
Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
total elements=67071935, nodata elements=8624611
largest temporary files:
FILL: 3.50G (3756028360) [67071935 elements, 56B each]
FLOW: 4.35G (4675785920) [58447324 elements, 80B each]
Will need at least 8.71G (9351571840) space available in /tmp

are you using a true 64bit OS?

EMPQUEUEADAPTIVE: starting in-memory pqueue
EMPQUEUEADAPTIVE: available memory: 297.929MB
EMPQUEUEADAPTIVE: desired memory: 297.929MB
sz_stream: 270388 buf_arity: 200 mm_overhead: 8665728
mm_avail: 312400802.
EMPQUEUEADAPTIVE: memory overhead set to 8.26428MB
EMPQUEUEADAPTIVE: pqsize set to 37966884
assigning preliminary directions
finding flat areas (plateaus and depressions)
file=/tmp/STREAM_rSqNkF:cannot read!: Bad address
r.terraflow:

/usr/local/svn/grass/grass640_rc5/dist.i686-pc-linux-gnu/include/grass/iostream/ami_sort_impl.h:91:
size_t makeRun_Block(AMI_STREAM<T>*, T*,
unsigned int, Compare*) [with T =
plateauType, Compare = ijCmpPlateauType]: Assertion `err ==
AMI_ERROR_NO_ERROR || err == AMI_ERROR_END_OF_STREAM' failed.
Abortito

(can't make a 8gig file on a 32bit OS/filesystem)

Hamish

Francesco_Mirabella · November 9, 2009, 2:06pm

Hi Markus and Hamish,
thanks for your message, this is the system I am on:

System-> Debian
Kernel-> 2.6.26.2
Ram-> 2.0 Gb
Clock-> Intel Pentium D 945 / 3.4 GHz 32/64 bit ( Dual-Core )
Free disk space-> 20 Gb on /

I tried increasing the memory to 1.6 Gb (about 80% of RAM) but I get the same error (below)

Can you tell how to use the .diff? Sorry but I am not familiar with them .... I'll be happy to test it and report what happens

Francesco

This is the output using 1.6Gb of memory

GRASS 6.4.0RC5 (WGS84_UTM33):~ > r.terraflow elevation=copia.dem filled=flood direction=flow swatershed=sink accumulation=accumulation tci=tci d8cut=infinity memory=1600 STREAM_DIR=/tmp stats=stats.out
STREAM temporary files in /tmp (THESE INTERMEDIATE STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES MANUALLY!)
file stats.out exists - renaming.
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 1.56G (1677721600) bytes
Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
total elements=67071935, nodata elements=8624611
largest temporary files:
FILL: 3.50G (3756028360) [67071935 elements, 56B each]
FLOW: 4.35G (4675785920) [58447324 elements, 80B each]
Will need at least 8.71G (9351571840) space available in /tmp
------------------------------
COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
file=/tmp/STREAM_H2x7Lf:cannot read!: Bad address
r.terraflow: /usr/local/svn/grass/grass640_rc5/dist.i686-pc-linux-gnu/include/grass/iostream/ami_sort_impl.h:91: size_t makeRun_Block(AMI_STREAM<T>, T, unsigned int, Compare*) [with T = nodataType, Compare = labelCmpNodataType]: Assertion `err == AMI_ERROR_NO_ERROR || err == AMI_ERROR_END_OF_STREAM' failed.
Abortito

Hamish wrote:

Francesco wrote:
----------------------------
GRASS 6.4.0RC5 (WGS84_UTM33):~ > r.terraflow elevation=copia.dem

filled=flood direction=flow swatershed=sink

accumulation=accumulation
tci=tci d8cut=infinity memory=300 STREAM_DIR=/tmp stats=stats.out

can you set memory= to something higher? (say 80% of physical RAM)

Memory size: 300.00M (314572800) bytes
Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
total elements=67071935, nodata elements=8624611
largest temporary files:
FILL: 3.50G (3756028360) [67071935 elements, 56B each]
FLOW: 4.35G (4675785920) [58447324 elements, 80B each]
Will need at least 8.71G (9351571840) space available in /tmp

are you using a true 64bit OS?

EMPQUEUEADAPTIVE: starting in-memory pqueue
EMPQUEUEADAPTIVE: available memory: 297.929MB
EMPQUEUEADAPTIVE: desired memory: 297.929MB
sz_stream: 270388 buf_arity: 200 mm_overhead: 8665728
mm_avail: 312400802.
EMPQUEUEADAPTIVE: memory overhead set to 8.26428MB
EMPQUEUEADAPTIVE: pqsize set to 37966884
assigning preliminary directions
finding flat areas (plateaus and depressions)
file=/tmp/STREAM_rSqNkF:cannot read!: Bad address
r.terraflow:

/usr/local/svn/grass/grass640_rc5/dist.i686-pc-linux-gnu/include/grass/iostream/ami_sort_impl.h:91:
size_t makeRun_Block(AMI_STREAM<T>*, T*,
unsigned int, Compare*) [with T =
plateauType, Compare = ijCmpPlateauType]: Assertion `err ==
AMI_ERROR_NO_ERROR || err == AMI_ERROR_END_OF_STREAM' failed.
Abortito

(can't make a 8gig file on a 32bit OS/filesystem)

Hamish

Francesco_Mirabella · November 10, 2009, 11:34am

Hi Markus,
I tried to apply the patch of r.terraflow. I do not know if I did it fine, what I did was to change the original "include/iostream/ami_sort_impl.h" file with the modifications in the .diff file. If this was fine, below is the output of r.terraflow which to me seems to give a different error as before.

===============================================================
GRASS 6.4.0RC5 (WGS84_UTM33):~ > r.terraflow elevation=dem_tagliato filled=flood direction=flow swatershed=sink accumulation=accumulation tci=tci d8cut=infinity memory=1600 STREAM_DIR=/tmp stats=stats.out
STREAM temporary files in /tmp (THESE INTERMEDIATE STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES MANUALLY!)
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 1.56G (1677721600) bytes
Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
total elements=67071935, nodata elements=61864898
largest temporary files:
FILL: 3.50G (3756028360) [67071935 elements, 56B each]
FLOW: 397.27M (416562960) [5207037 elements, 80B each]
Will need at least 7.00G (7512056720) space available in /tmp
------------------------------
COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
EMPQUEUEADAPTIVE: starting in-memory pqueue
EMPQUEUEADAPTIVE: available memory: 1597.93MB
EMPQUEUEADAPTIVE: desired memory: 1597.93MB
sz_stream: 270388 buf_arity: 200 mm_overhead: 8665728 mm_avail: 1675549602.
EMPQUEUEADAPTIVE: memory overhead set to 8.26428MB
EMPQUEUEADAPTIVE: pqsize set to 208360484
assigning preliminary directions
finding flat areas (plateaus and depressions)
EMPQUEUEADAPTIVE: starting in-memory pqueue
EMPQUEUEADAPTIVE: available memory: 1597.41MB
EMPQUEUEADAPTIVE: desired memory: 1597.41MB
sz_stream: 270388 buf_arity: 200 mm_overhead: 8665728 mm_avail: 1675008754.
EMPQUEUEADAPTIVE: memory overhead set to 8.26428MB
EMPQUEUEADAPTIVE: pqsize set to 208292878
assigning directions on plateaus
generating watersheds and watershed graph
AMI_STREAM::write_item failed.
/tmp/STREAM_HHSoQw: File too large
r.terraflow: /usr/local/svn/grass/grass640_rc5/dist.i686-pc-linux-gnu/include/grass/iostream/ami_stream.h:560: AMI_err AMI_STREAM<T>::write_item(const T&) [with T = compressedWaterWindowType]: Assertion `0' failed.
Abortito
GRASS 6.4.0RC5 (WGS84_UTM33):~ >

Francesco

Markus Neteler wrote:

On Mon, Nov 9, 2009 at 1:24 PM, Francesco Mirabella <mirabell@unipg.it> wrote:

Hi all,
I am trying to get flow directions out of a dem (10m resolution). I have
tried r.terraflow which gives me the error below:
Can anyone tell me if I am doing something wrong and how can I solve this?
many thanks
Francesco
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
GRASS 6.4.0RC5 (WGS84_UTM33):~ > r.terraflow elevation=copia.dem
filled=flood direction=flow swatershed=sink accumulation=accumulation
tci=tci d8cut=infinity memory=300 STREAM_DIR=/tmp stats=stats.out

STREAM temporary files in /tmp (THESE INTERMEDIATE STREAMS WILL NOT BE
DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE
DELETE THESE FILES MANUALLY!)
file stats.out exists - renaming.
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 300.00M (314572800) bytes
Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
total elements=67071935, nodata elements=8624611
largest temporary files:
FILL: 3.50G (3756028360) [67071935 elements, 56B each]
FLOW: 4.35G (4675785920) [58447324 elements, 80B each]
Will need at least 8.71G (9351571840) space available in /tmp
------------------------------
COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
EMPQUEUEADAPTIVE: starting in-memory pqueue
EMPQUEUEADAPTIVE: available memory: 297.929MB
EMPQUEUEADAPTIVE: desired memory: 297.929MB
sz_stream: 270388 buf_arity: 200 mm_overhead: 8665728 mm_avail: 312400802.
EMPQUEUEADAPTIVE: memory overhead set to 8.26428MB
EMPQUEUEADAPTIVE: pqsize set to 37966884
assigning preliminary directions
finding flat areas (plateaus and depressions)
file=/tmp/STREAM_rSqNkF:cannot read!: Bad address
r.terraflow:
/usr/local/svn/grass/grass640_rc5/dist.i686-pc-linux-gnu/include/grass/iostream/ami_sort_impl.h:91:
size_t makeRun_Block(AMI_STREAM<T>*, T*, unsigned int, Compare*) [with T =
plateauType, Compare = ijCmpPlateauType]: Assertion `err ==
AMI_ERROR_NO_ERROR || err == AMI_ERROR_END_OF_STREAM' failed.
Abortito

This is a known bug:
http://trac.osgeo.org/grass/ticket/775

with a patch (r.terraflow.diff) to test. Perhaps you could try it
and report back directly in the ticket?

thanks
Markus

--
**********************************************
Francesco Mirabella,
Geologia Strutturale e Geofisica
Universita' di Perugia,
Dipartimento di Scienze della Terra,
Piazza Universita' 1, 06100 Perugia (Italy)
tel: ++39.(0)75.584.7948
fax: ++39.(0)75.585.2603
skype: francesco.mirabella
web: http://www.unipg.it/~mirabell/
**********************************************

neteler · November 11, 2009, 7:04am

Hi Francesco,

On Tue, Nov 10, 2009 at 12:34 PM, Francesco Mirabella <mirabell@unipg.it> wrote:

Hi Markus,
I tried to apply the patch of r.terraflow. I do not know if I did it fine,
what I did was to change the original "include/iostream/ami_sort_impl.h"
file with the modifications in the .diff file.

such changes can be applied automatically:

cd where/the/code/is/
patch -p0 < difffile

Sometimes it also needs to be applied from the main GRASS source
code directory (just check if the path is in the diff file or not).

If this was fine, below is
the output of r.terraflow which to me seems to give a different error as
before.

===============================================================
GRASS 6.4.0RC5 (WGS84_UTM33):~ > r.terraflow elevation=dem_tagliato
filled=flood direction=flow swatershed=sink accumulation=accumulation
tci=tci d8cut=infinity memory=1600 STREAM_DIR=/tmp stats=stats.out
STREAM temporary files in /tmp (THESE INTERMEDIATE STREAMS WILL NOT BE
DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE
DELETE THESE FILES MANUALLY!)
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 1.56G (1677721600) bytes
Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
total elements=67071935, nodata elements=61864898
largest temporary files:
FILL: 3.50G (3756028360) [67071935 elements, 56B each]

-> are you on a 64 box? If on 32bit, did you compile GRASS with
large file support? There will be the 2GB limit if not.

FLOW: 397.27M (416562960) [5207037 elements, 80B each]
Will need at least 7.00G (7512056720) space available in /tmp

-> space is there in /tmp/?

------------------------------
COMPUTING FLOW DIRECTIONS

...

generating watersheds and watershed graph
AMI_STREAM::write_item failed.
/tmp/STREAM_HHSoQw: File too large

-> /tmp full or 2GB file limit hit?

Please check if you configured GRASS with --enable-largefile before
compilation.

Markus

Francesco_Mirabella · November 11, 2009, 11:53am

Hi Markus,
I am new to code patching, yes there is space in the /tmp directory (about 14Gb) :
------------------------------------------------------------
francesco@terra17:~$ df -h /tmp/
Filesystem Dimens. Usati Disp. Uso% Montato su
/dev/sdb1 33G 18G 14G 57% /
------------------------------------------------------------

I have now re-configured with both 64bit and large file support.
I get the same error however I need e clarification:

There exist two ami_sort_impl.h files, one in
/../../../grass640_rc5/dist.i686-pc-linux-gnu/include/grass/iostream/
and the other in
/../../../grass640_rc5/include/iostream/

I have tried to apply the patch to the first, then to second and to both. However, I get very similar errors to the original error when using the patch for the first and for the second and for both. I attach four files with the errors for the original and with the patched (first, second and both).
Is this helpful? any hints?

grazie
Francesco

Markus Neteler wrote:

Hi Francesco,

On Tue, Nov 10, 2009 at 12:34 PM, Francesco Mirabella <mirabell@unipg.it> wrote:

Hi Markus,
I tried to apply the patch of r.terraflow. I do not know if I did it fine,
what I did was to change the original "include/iostream/ami_sort_impl.h"
file with the modifications in the .diff file.

such changes can be applied automatically:

cd where/the/code/is/
patch -p0 < difffile

Sometimes it also needs to be applied from the main GRASS source
code directory (just check if the path is in the diff file or not).

If this was fine, below is
the output of r.terraflow which to me seems to give a different error as
before.

===============================================================
GRASS 6.4.0RC5 (WGS84_UTM33):~ > r.terraflow elevation=dem_tagliato
filled=flood direction=flow swatershed=sink accumulation=accumulation
tci=tci d8cut=infinity memory=1600 STREAM_DIR=/tmp stats=stats.out
STREAM temporary files in /tmp (THESE INTERMEDIATE STREAMS WILL NOT BE
DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE
DELETE THESE FILES MANUALLY!)
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 1.56G (1677721600) bytes
Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
total elements=67071935, nodata elements=61864898
largest temporary files:
FILL: 3.50G (3756028360) [67071935 elements, 56B each]

-> are you on a 64 box? If on 32bit, did you compile GRASS with
large file support? There will be the 2GB limit if not.

FLOW: 397.27M (416562960) [5207037 elements, 80B each]
Will need at least 7.00G (7512056720) space available in /tmp

-> space is there in /tmp/?

------------------------------
COMPUTING FLOW DIRECTIONS

...

generating watersheds and watershed graph
AMI_STREAM::write_item failed.
/tmp/STREAM_HHSoQw: File too large

-> /tmp full or 2GB file limit hit?

Please check if you configured GRASS with --enable-largefile before
compilation.

Markus

(attachments)

r.terraflow_both_patched.txt (2.33 KB)
r.terraflow_dist_file_patched.txt (2.33 KB)
r.terraflow_original_code.txt (2.33 KB)
r.terraflow_source_file_patched.txt (2.33 KB)

neteler · November 11, 2009, 12:08pm

Hi Francesco,

On Wed, Nov 11, 2009 at 12:53 PM, Francesco Mirabella <mirabell@unipg.it> wrote:

Hi Markus,
I am new to code patching, yes there is space in the /tmp directory
(about 14Gb) :

..

I have now re-configured with both 64bit and large file support.

well, did you are recompile?

I get the same error however I need e clarification:

There exist two ami_sort_impl.h files, one in
/../../../grass640_rc5/dist.i686-pc-linux-gnu/include/grass/iostream/

... that's not relevant.

and the other in
/../../../grass640_rc5/include/iostream/

... that's relevant.

I have tried to apply the patch to the first, then to second and to both.
However, I get very similar errors to the original error when using the
patch for the first and for the second and for both.

Unless you don't recompile it won't work.
The "first" instance above is a copy created during compilation
of the "second" instance which is the real code.

1. Patch
grass640_rc5/include/iostream/

2. configure

3. compile

4. install or run directly from GRASS from
grass640_rc5/bin.i686-pc-linux-gnu/grass64

All clear like mud?

Markus

Francesco_Mirabella · November 11, 2009, 1:43pm

Hi Markus,
thanks for you reply, it seems that the patch works.... below I attach the command result
Now I'll try to play with the maps .....
many thanks
Francesco

============================================
GRASS 6.4.0RC5 (WGS84_UTM33):~ > r.terraflow elevation=dem_tagliato filled=flood direction=flow swatershed=sink accumulation=accumulation tci=tci d8cut=infinity memory=1600 STREAM_DIR=/tmp stats=stats.out
STREAM temporary files in /tmp (THESE INTERMEDIATE STREAMS WILL NOT BE DELETED IN CASE OF ABNORMAL TERMINATION OF THE PROGRAM. TO SAVE SPACE PLEASE DELETE THESE FILES MANUALLY!)
MFD flow direction
D8CUT=999999986991104.000000
Memory size: 1.56G (1677721600) bytes
Memory manager registering memory in MM_IGNORE_MEMORY_EXCEEDED mode.
total elements=67071935, nodata elements=61864898
largest temporary files:
FILL: 3.50G (3756028360) [67071935 elements, 56B each]
FLOW: 397.27M (416562960) [5207037 elements, 80B each]
Will need at least 7.00G (7512056720) space available in /tmp
------------------------------
COMPUTING FLOW DIRECTIONS
classifying nodata (inner & boundary)
EMPQUEUEADAPTIVE: starting in-memory pqueue
EMPQUEUEADAPTIVE: available memory: 1597.93MB
EMPQUEUEADAPTIVE: desired memory: 1597.93MB
sz_stream: 270388 buf_arity: 200 mm_overhead: 8665728 mm_avail: 1675549602.
EMPQUEUEADAPTIVE: memory overhead set to 8.26428MB
EMPQUEUEADAPTIVE: pqsize set to 208360484
assigning preliminary directions
finding flat areas (plateaus and depressions)
EMPQUEUEADAPTIVE: starting in-memory pqueue
EMPQUEUEADAPTIVE: available memory: 1597.41MB
EMPQUEUEADAPTIVE: desired memory: 1597.41MB
sz_stream: 270388 buf_arity: 200 mm_overhead: 8665728 mm_avail: 1675008754.
EMPQUEUEADAPTIVE: memory overhead set to 8.26428MB
EMPQUEUEADAPTIVE: pqsize set to 208292878
assigning directions on plateaus
generating watersheds and watershed graph
EMPQUEUEADAPTIVE: starting in-memory pqueue
EMPQUEUEADAPTIVE: available memory: 1598.96MB
EMPQUEUEADAPTIVE: desired memory: 1598.96MB
sz_stream: 270396 buf_arity: 200 mm_overhead: 8678784 mm_avail: 1676631202.
EMPQUEUEADAPTIVE: memory overhead set to 8.27673MB
EMPQUEUEADAPTIVE: pqsize set to 104247026
flooding depressions
available memory: 1599MB (1676901678B)
UnionFind::makeSet: reallocate double 2000
warning: watershed 1 (R=1) not done
------------------------------
REASSIGNING DIRECTIONS
finding flat areas (plateaus and depressions)
EMPQUEUEADAPTIVE: starting in-memory pqueue
EMPQUEUEADAPTIVE: available memory: 1597.15MB
EMPQUEUEADAPTIVE: desired memory: 1597.15MB
sz_stream: 270388 buf_arity: 200 mm_overhead: 8665728 mm_avail: 1674738378.
EMPQUEUEADAPTIVE: memory overhead set to 8.26428MB
EMPQUEUEADAPTIVE: pqsize set to 208259081
EMPQUEUEADAPTIVE: starting in-memory pqueue
EMPQUEUEADAPTIVE: available memory: 1596.38MB
EMPQUEUEADAPTIVE: desired memory: 1596.38MB
sz_stream: 270388 buf_arity: 200 mm_overhead: 8665728 mm_avail: 1673927126.
EMPQUEUEADAPTIVE: memory overhead set to 8.26428MB
EMPQUEUEADAPTIVE: pqsize set to 208157674
assigning directions on plateaus
creating flowStream: [AMI_STREAM /tmp/flowStream 0]
compute flow directions done.
writing stream /tmp/STREAM_fE0Ows to raster map flow: 100%
writing stream /tmp/STREAM_G82bHw to raster map flood: 100%
writing stream /tmp/STREAM_tSF3ML to raster map sink: 100%
------------------------------
COMPUTING FLOW ACCUMULATION
creating sweep stream from fill output stream
sorting sweep stream
sweeping: EMPQUEUEADAPTIVE: starting in-memory pqueue
EMPQUEUEADAPTIVE: available memory: 1599.73MB
EMPQUEUEADAPTIVE: desired memory: 1599.73MB
sz_stream: 270396 buf_arity: 200 mm_overhead: 8678784 mm_avail: 1677442317.
EMPQUEUEADAPTIVE: memory overhead set to 8.27673MB
EMPQUEUEADAPTIVE: pqsize set to 104297720
100%
sorting sweep output stream
writing stream /tmp/STREAM_a2UrtD to raster maps accumulation, tci: 100%
r.terraflow complete.
GRASS 6.4.0RC5 (WGS84_UTM33):~ >

Markus Neteler wrote:

Hi Francesco,

On Wed, Nov 11, 2009 at 12:53 PM, Francesco Mirabella <mirabell@unipg.it> wrote:

Hi Markus,
I am new to code patching, yes there is space in the /tmp directory
(about 14Gb) :

..

I have now re-configured with both 64bit and large file support.

well, did you are recompile?

I get the same error however I need e clarification:

There exist two ami_sort_impl.h files, one in
/../../../grass640_rc5/dist.i686-pc-linux-gnu/include/grass/iostream/

... that's not relevant.

and the other in
/../../../grass640_rc5/include/iostream/

... that's relevant.

I have tried to apply the patch to the first, then to second and to both.
However, I get very similar errors to the original error when using the
patch for the first and for the second and for both.

Unless you don't recompile it won't work.
The "first" instance above is a copy created during compilation
of the "second" instance which is the real code.

1. Patch
grass640_rc5/include/iostream/

2. configure

3. compile

4. install or run directly from GRASS from
grass640_rc5/bin.i686-pc-linux-gnu/grass64

All clear like mud?

Markus

--
**********************************************
Francesco Mirabella,
Geologia Strutturale e Geofisica
Universita' di Perugia,
Dipartimento di Scienze della Terra,
Piazza Universita' 1, 06100 Perugia (Italy)
tel: ++39.(0)75.584.7948
fax: ++39.(0)75.585.2603
skype: francesco.mirabella
web: http://www.unipg.it/~mirabell/
**********************************************

neteler · November 11, 2009, 1:59pm

On Wed, Nov 11, 2009 at 2:43 PM, Francesco Mirabella <mirabell@unipg.it> wrote:

Hi Markus,
thanks for you reply, it seems that the patch works.... below I attach the
command result
Now I'll try to play with the maps .....
many thanks
Francesco

Excellent. I have submitted the patch to GRASS 6.4, 6.5 and 7.

Markus