[GRASS-user] r.hazard.flood: ERROR_G_malloc: r_accumulation

RichTC · June 10, 2013, 1:54pm

Hi,

Maybe some one could kindly advise what I'm doing wrong here.

Previously I successfully used r.hazard.flood to generate flood and mti
layers. However, now I'm encountering the following error with two other
different DEM layers. The error appears to be related to memory, but there
is plenty of RAM free (6GB) and I still get the error with a DEM subset of
only 5MB in size. Also, I'm able to repeat the r.hazard.flood process
successfully using the first DEM I tried, so it would seem to be related to
the DEMs?

I've set g.region - selecting the raster file to be used with r.hazard.flood
and resolution (in meters).

Thanks for your suggestions,

Richard

r.hazard.flood.py map=ASTGTM_composite_all_de@PERMANENT
flood=ASTGTM_composite_all_delta_flood mti=ASTGTM_composite_all_delta_mti
Cellsize : 30.0
SECTION 1a (of 4): Initiating Memory.
Current region rows: 17981, cols: 11271
ERROR: G_malloc: unable to allocate 811437852 bytes of memory at
init_vars.c:136
WARNING: Subprocess failed with exit code 256
WARNING: category information for [r_accumulation] in [PERMANENT] missing or
invalid
Flow accumulation done.
Slope raster map <r_slope> complete
Slope map done.
Exponent : 0.076488432313
MTI threshold : 3.11495902789
Calculating mti raster map..
Invalid map <r_accumulation>
Parse error
ERROR: An error occurred while running r.mapcalc

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/r-hazard-flood-ERROR-G-malloc-r-accumulation-tp5059130.html
Sent from the Grass - Users mailing list archive at Nabble.com.

neteler · June 10, 2013, 1:59pm

On Mon, Jun 10, 2013 at 3:54 PM, RichardC <richtcooper@hotmail.com> wrote:

Hi,

Maybe some one could kindly advise what I'm doing wrong here.

Previously I successfully used r.hazard.flood to generate flood and mti
layers. However, now I'm encountering the following error with two other
different DEM layers. The error appears to be related to memory, but there

...

r.hazard.flood.py map=ASTGTM_composite_all_de@PERMANENT
flood=ASTGTM_composite_all_delta_flood mti=ASTGTM_composite_all_delta_mti
Cellsize : 30.0
SECTION 1a (of 4): Initiating Memory.
Current region rows: 17981, cols: 11271
ERROR: G_malloc: unable to allocate 811437852 bytes of memory at
init_vars.c:136

I guess that you are exceeding the 32bit specifications of your PC, if
- you are on a 32bit PC
- without Large file Support enabled (note that it is better
implemented in GRASS 7)

How to find out:

17981 * 11271

[1] 202663851

2^31

[1] 2147483648

# check
uname -a

... and tell us which GRASS Version you use on which system.

Also check if the computation region is as desired (g.region).

hope this helps
Markus

Markus_Metz · June 10, 2013, 7:17pm

On Mon, Jun 10, 2013 at 3:54 PM, RichardC <richtcooper@hotmail.com> wrote:

Hi,

Maybe some one could kindly advise what I'm doing wrong here.

Previously I successfully used r.hazard.flood to generate flood and mti
layers. However, now I'm encountering the following error with two other
different DEM layers. The error appears to be related to memory, but there
is plenty of RAM free (6GB) and I still get the error with a DEM subset of
only 5MB in size. Also, I'm able to repeat the r.hazard.flood process
successfully using the first DEM I tried, so it would seem to be related to
the DEMs?

I've set g.region - selecting the raster file to be used with r.hazard.flood
and resolution (in meters).

Thanks for your suggestions,

Richard

r.hazard.flood.py map=ASTGTM_composite_all_de@PERMANENT
flood=ASTGTM_composite_all_delta_flood mti=ASTGTM_composite_all_delta_mti
Cellsize : 30.0
SECTION 1a (of 4): Initiating Memory.
Current region rows: 17981, cols: 11271
ERROR: G_malloc: unable to allocate 811437852 bytes of memory at
init_vars.c:136

You could modify the r.hazard.flood.py python script such that it
calls r.watershed with the memory-safe option,: replace

grass.run_command('r.watershed', elevation = r_elevation ,
accumulation = 'r_accumulation' , convergence = 5, flags = 'a')

with

grass.run_command('r.watershed', elevation = r_elevation ,
accumulation = 'r_accumulation' , convergence = 5, flags = 'am')

HTH,

Markus M

WARNING: Subprocess failed with exit code 256
WARNING: category information for [r_accumulation] in [PERMANENT] missing or
invalid
Flow accumulation done.
Slope raster map <r_slope> complete
Slope map done.
Exponent : 0.076488432313
MTI threshold : 3.11495902789
Calculating mti raster map..
Invalid map <r_accumulation>
Parse error
ERROR: An error occurred while running r.mapcalc

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/r-hazard-flood-ERROR-G-malloc-r-accumulation-tp5059130.html
Sent from the Grass - Users mailing list archive at Nabble.com.
_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

RichTC · June 11, 2013, 4:36am

Thanks for your suggestions.

I modified the following:

/home/user/.grass6/addons/scripts/r.hazard.flood, replacing:

# Flow accumulation map MFD
    grass.run_command('r.watershed', elevation = r_elevation , accumulation
= 'r_accumulation' , convergence = 5, flags = 'fa')
    grass.message("Flow accumulation done. ")

with

  # Flow accumulation map MFD
    grass.run_command('r.watershed', elevation = r_elevation , accumulation
= 'r_accumulation' , convergence = 5, flags = 'am')
    grass.message("Flow accumulation done. ")

This enabled me run r.hazard.flow on a portion (sub-basin) of the DEM. I ran
into trouble if I attempted the larger mosaic I created, as follows:

r.hazard.flood.py map=ASTGTM_composite_all@PERMANENT
flood=ASTGTM_composite_all_flood mti=ASTGTM_composite_all_mti
Cellsize : 30.0
SECTION 1 beginning: Initiating Variables. 4 sections total.
WARNING: segment zero_fill(): Unable to write (No such file or directory)
WARNING: seg_open(): could not write segment file
SECTION 1b (of 4): Determining Offmap Flow.
Floating point exception (core dumped)
WARNING: Subprocess failed with exit code 34816
WARNING: category information for [r_accumulation] in [PERMANENT] missing or
invalid
Flow accumulation done.
Slope raster map <r_slope> complete
Slope map done.
Exponent : 0.076488432313
MTI threshold : 3.11495902789
Calculating mti raster map..
Invalid map <r_accumulation>
Parse error
ERROR: An error occurred while running r.mapcalc
(Tue Jun 11 10:55:33 2013) Command finished (9 min 3 sec)

I guess the workaround solution is to reduce the spatial extent and run each
sub-basin separately with r.hazard.flood (if possible for all I'm not sure)

Install GRASS 7 - installing from source into my home directory I think
should allow two systems to work on my machine ...

My current system:

uname -a: Linux 3.2.0-31-generic-pae #50-Ubuntu SMP Fri Sep 7 16:39:45 UTC
2012 i686 i686 i386 GNU/Linux

I'm running Linux Mint 13 (Maya) which is based on Ubuntu Precise (installed
system wide via package manager)

GRASS version: 6.4.3 (Python 2.7.3/wxpython 2.8.12.1)
GRASS: 6.4.3-1~svn54876~precise1

NB: I just noticed that the repo has been updated with 6.4.3 (RC3-4) ...

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/r-hazard-flood-ERROR-G-malloc-r-accumulation-tp5059130p5059281.html
Sent from the Grass - Users mailing list archive at Nabble.com.

Markus_Metz · June 11, 2013, 9:36am

On Tue, Jun 11, 2013 at 6:36 AM, RichardC <richtcooper@hotmail.com> wrote:

Thanks for your suggestions.

I modified the following:

/home/user/.grass6/addons/scripts/r.hazard.flood, replacing:

# Flow accumulation map MFD
    grass.run_command('r.watershed', elevation = r_elevation , accumulation
= 'r_accumulation' , convergence = 5, flags = 'fa')
    grass.message("Flow accumulation done. ")

with

  # Flow accumulation map MFD
    grass.run_command('r.watershed', elevation = r_elevation , accumulation
= 'r_accumulation' , convergence = 5, flags = 'am')

For GRASS 6, you should keep the -f flag for MFD.

grass.message("Flow accumulation done. ")

This enabled me run r.hazard.flow on a portion (sub-basin) of the DEM. I ran
into trouble if I attempted the larger mosaic I created, as follows:

r.hazard.flood.py map=ASTGTM_composite_all@PERMANENT
flood=ASTGTM_composite_all_flood mti=ASTGTM_composite_all_mti
Cellsize : 30.0
SECTION 1 beginning: Initiating Variables. 4 sections total.
WARNING: segment zero_fill(): Unable to write (No such file or directory)

This warning indicates that either you were running out of disk space
or the temporary file would be larger than supported (GRASS 6 has no
global Large File Support), or both.

WARNING: seg_open(): could not write segment file
SECTION 1b (of 4): Determining Offmap Flow.
Floating point exception (core dumped)
WARNING: Subprocess failed with exit code 34816
WARNING: category information for [r_accumulation] in [PERMANENT] missing or
invalid
Flow accumulation done.
Slope raster map <r_slope> complete
Slope map done.
Exponent : 0.076488432313
MTI threshold : 3.11495902789
Calculating mti raster map..
Invalid map <r_accumulation>
Parse error
ERROR: An error occurred while running r.mapcalc
(Tue Jun 11 10:55:33 2013) Command finished (9 min 3 sec)

I guess the workaround solution is to reduce the spatial extent and run each
sub-basin separately with r.hazard.flood (if possible for all I'm not sure)

Install GRASS 7 - installing from source into my home directory I think
should allow two systems to work on my machine ...

That would work, I have 4 different GRASS versions running on my
machine. Note that you do not need to run 'make install' in order to
use GRASS.

My current system:

uname -a: Linux 3.2.0-31-generic-pae #50-Ubuntu SMP Fri Sep 7 16:39:45 UTC
2012 i686 i686 i386 GNU/Linux

You are using a 32 bit system (why?), therefore you need to configure
GRASS with --enable-largefile.

HTH,

Markus M

I'm running Linux Mint 13 (Maya) which is based on Ubuntu Precise (installed
system wide via package manager)

GRASS version: 6.4.3 (Python 2.7.3/wxpython 2.8.12.1)
GRASS: 6.4.3-1~svn54876~precise1

NB: I just noticed that the repo has been updated with 6.4.3 (RC3-4) ...

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/r-hazard-flood-ERROR-G-malloc-r-accumulation-tp5059130p5059281.html
Sent from the Grass - Users mailing list archive at Nabble.com.
_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

RichTC · June 14, 2013, 7:19am

Just to confirm (and for reference) the successful running of r.hazard.flood
on GRASS 7 svn (code revision 56683) (configured with --enable-largefile),
and with the following 'flags' change to r.hazard.flood.py:

from
    # Flow accumulation map MFD
    grass.run_command('r.watershed', elevation = r_elevation , accumulation
= 'r_accumulation' , convergence = 5, flags = 'a')
    grass.message("Flow accumulation done. ")

to
    # Flow accumulation map MFD
    grass.run_command('r.watershed', elevation = r_elevation , accumulation
= 'r_accumulation' , convergence = 5, flags = 'am')
    grass.message("Flow accumulation done. ")

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/r-hazard-flood-ERROR-G-malloc-r-accumulation-tp5059130p5060063.html
Sent from the Grass - Users mailing list archive at Nabble.com.

Markus_Metz · June 14, 2013, 7:58am

On Fri, Jun 14, 2013 at 9:19 AM, RichardC <richtcooper@hotmail.com> wrote:

Just to confirm (and for reference) the successful running of r.hazard.flood
on GRASS 7 svn (code revision 56683) (configured with --enable-largefile),
and with the following 'flags' change to r.hazard.flood.py:

from
    # Flow accumulation map MFD
    grass.run_command('r.watershed', elevation = r_elevation , accumulation
= 'r_accumulation' , convergence = 5, flags = 'a')
    grass.message("Flow accumulation done. ")

to
    # Flow accumulation map MFD
    grass.run_command('r.watershed', elevation = r_elevation , accumulation
= 'r_accumulation' , convergence = 5, flags = 'am')
    grass.message("Flow accumulation done. ")

Nice, thanks for the feedback!

Markus M