[GRASS-dev] r.terraflow broken

GRASS GRASS developer
1

It appears to be related to memory management:

  $ r.terraflow help
  MM error: limit =0B. allocating 24B. limit exceeded by 32B.
  r.terraflow: mm.cc:344: void* operator new(size_t): Assertion `0' failed.
  Aborted (core dumped)

--
Glynn Clements <glynn@gclements.plus.com>

2

Hi Laura,

we have new problem in GRASS 7 with r.terraflow:

On Mon, Aug 3, 2009 at 8:40 PM, Glynn Clements<glynn@gclements.plus.com> wrote:

It appears to be related to memory management:

   $ r\.terraflow help
   MM error: limit =0B\. allocating 24B\. limit exceeded by 32B\.
   r\.terraflow: mm\.cc:344: void\* operator new\(size\_t\): Assertion \`0&#39; failed\.
   Aborted \(core dumped\)

Happens at least on Gentoo (?) and Mandriva.

But
grass64/lib/iostream/mm.cc
and
grass70/lib/iostream/mm.cc
look the same despite the commented header.

Any idea?

thanks
Markus

3

The problem is that it attempts to set the memory manager limit to 0 (see below: limit=0B), and the first line in
MM_register::set_memory_limit(size_t new_limit)

is

assert(new_limit>0).

Therefore the problem.
So the question is, why does it try to set the memory limit to 0?
Unless the user specifies a value, the default value is 300MB, see main.cc:

mem->answer="300";
...
opt->mem = atoi(mem->answer);
..
size_t mm_size = opt->mem << 20; /* opt->mem is in MB */
MM_manager.set_memory_limit(mm_size);

Did you use the default value of memory, or a different one?
I wonder what size_t is on that platform.

-Laura

On Aug 3, 2009, at 2:47 PM, Markus Neteler wrote:

Hi Laura,

we have new problem in GRASS 7 with r.terraflow:

On Mon, Aug 3, 2009 at 8:40 PM, Glynn Clements<glynn@gclements.plus.com> wrote:

It appears to be related to memory management:

       $ r.terraflow help
       MM error: limit =0B. allocating 24B. limit exceeded by 32B.
       r.terraflow: mm.cc:344: void* operator new(size_t): Assertion `0' failed.
       Aborted (core dumped)

Happens at least on Gentoo (?) and Mandriva.

But
grass64/lib/iostream/mm.cc
and
grass70/lib/iostream/mm.cc
look the same despite the commented header.

Any idea?

thanks
Markus

4

Laura Toma wrote:

The problem is that it attempts to set the memory manager limit to 0
(see below: limit=0B), and the first line in
MM_register::set_memory_limit(size_t new_limit)

is

assert(new_limit>0).

Therefore the problem.
So the question is, why does it try to set the memory limit to 0?
Unless the user specifies a value, the default value is 300MB, see
main.cc:

mem->answer="300";
...
opt->mem = atoi(mem->answer);
..
size_t mm_size = opt->mem << 20; /* opt->mem is in MB */
MM_manager.set_memory_limit(mm_size);

It doesn't even get as far as main(), let alone parsing and using the
mem= option:

$ gdb r.terraflow

Program received signal SIGABRT, Aborted.
[Switching to Thread 0xb6bcf9a0 (LWP 668)]
0xb7fd4424 in __kernel_vsyscall ()

where

#0 0xb7fd4424 in __kernel_vsyscall ()
#1 0xb7d42311 in *__GI_raise (sig=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#2 0xb7d43bf8 in *__GI_abort () at abort.c:88
#3 0xb7d3b6df in *__GI___assert_fail (
    assertion=0x6 <Address 0x6 out of bounds>,
    file=0x6 <Address 0x6 out of bounds>, line=6,
    function=0x80e3985 "void* operator new(size_t)") at assert.c:78
#4 0x080c8985 in operator new (sz=24) at mm.cc:344
#5 0xb6ec7c2f in global constructors keyed to _ZN4geos7ProfileC2ESs ()
    at ../util/Profiler.cpp:20
#6 0xb6ec8165 in __do_global_ctors_aux () from /usr/lib/libgeos.so.2
#7 0xb6e3a3c9 in _init () from /usr/lib/libgeos.so.2
#8 0xb7fe3a77 in call_init (l=0xb6ef3108, argc=2, argv=0xbfead4c4,
    env=0xbfead4d0) at dl-init.c:70
#9 0xb7fe3b16 in _dl_init (main_map=0xb7ff2658, argc=2, argv=0xbfead4c4,
    env=0xbfead4d0) at dl-init.c:134
#10 0xb7fd59df in _dl_start_user () from /lib/ld-linux.so.2

The assert is triggering in the constructor for a static variable
within the GEOS library (linked in by GDAL).

geos-2.2.3/source/util/Profiler.cpp has:

  static Profiler *internal_profiler = new Profiler();

This gets called before main(), but the customised "operator new()" in
lib/iostream doesn't work until the memory manager has been
initialised, which doesn't occur until G_parser() has been called.

--
Glynn Clements <glynn@gclements.plus.com>

5

Can we just initialize the MM_manager to ignore memory limits until we have a chance
to set and enforce a limit? The change below allowed me to at least compile and run the help

— lib/iostream/mm.cc (revision 38633)
+++ lib/iostream/mm.cc (working copy)
@@ -458,7 +458,7 @@
MM_register MM_manager;
int MM_register::instances = 0; // Number of instances. (init)
// TPIE’s “register memory requests” flag
-MM_mode MM_register::register_new = MM_ABORT_ON_MEMORY_EXCEEDED;
+MM_mode MM_register::register_new = MM_IGNORE_MEMORY_EXCEEDED;

On Wed, Aug 5, 2009 at 6:14 PM, Glynn Clements <glynn@gclements.plus.com> wrote:

Laura Toma wrote:

The problem is that it attempts to set the memory manager limit to 0
(see below: limit=0B), and the first line in
MM_register::set_memory_limit(size_t new_limit)

is

assert(new_limit>0).

Therefore the problem.
So the question is, why does it try to set the memory limit to 0?
Unless the user specifies a value, the default value is 300MB, see
main.cc:

mem->answer=“300”;
…
opt->mem = atoi(mem->answer);
…
size_t mm_size = opt->mem << 20; /* opt->mem is in MB */
MM_manager.set_memory_limit(mm_size);

It doesn’t even get as far as main(), let alone parsing and using the
mem= option:

$ gdb r.terraflow

Program received signal SIGABRT, Aborted.
[Switching to Thread 0xb6bcf9a0 (LWP 668)]
0xb7fd4424 in __kernel_vsyscall ()

where
#0 0xb7fd4424 in __kernel_vsyscall ()
#1 0xb7d42311 in *__GI_raise (sig=6)
at …/nptl/sysdeps/unix/sysv/linux/raise.c:64
#2 0xb7d43bf8 in *__GI_abort () at abort.c:88
#3 0xb7d3b6df in __GI___assert_fail (
assertion=0x6 <Address 0x6 out of bounds>,
file=0x6 <Address 0x6 out of bounds>, line=6,
function=0x80e3985 "void operator new(size_t)") at assert.c:78
#4 0x080c8985 in operator new (sz=24) at mm.cc:344
#5 0xb6ec7c2f in global constructors keyed to _ZN4geos7ProfileC2ESs ()
at …/util/Profiler.cpp:20
#6 0xb6ec8165 in __do_global_ctors_aux () from /usr/lib/libgeos.so.2
#7 0xb6e3a3c9 in _init () from /usr/lib/libgeos.so.2
#8 0xb7fe3a77 in call_init (l=0xb6ef3108, argc=2, argv=0xbfead4c4,
env=0xbfead4d0) at dl-init.c:70
#9 0xb7fe3b16 in _dl_init (main_map=0xb7ff2658, argc=2, argv=0xbfead4c4,
env=0xbfead4d0) at dl-init.c:134
#10 0xb7fd59df in _dl_start_user () from /lib/ld-linux.so.2

The assert is triggering in the constructor for a static variable
within the GEOS library (linked in by GDAL).

geos-2.2.3/source/util/Profiler.cpp has:

static Profiler *internal_profiler = new Profiler();

This gets called before main(), but the customised “operator new()” in
lib/iostream doesn’t work until the memory manager has been
initialised, which doesn’t occur until G_parser() has been called.

–

Glynn Clements <glynn@gclements.plus.com>

grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

6

Andrew Danner wrote:

Can we just initialize the MM_manager to ignore memory limits until we have
a chance
to set and enforce a limit? The change below allowed me to at least compile
and run the help

--- lib/iostream/mm.cc (revision 38633)
+++ lib/iostream/mm.cc (working copy)
@@ -458,7 +458,7 @@
MM_register MM_manager;
int MM_register::instances = 0; // Number of instances. (init)
// TPIE's "register memory requests" flag
-MM_mode MM_register::register_new = MM_ABORT_ON_MEMORY_EXCEEDED;
+MM_mode MM_register::register_new = MM_IGNORE_MEMORY_EXCEEDED;

Committed as r38702.

--
Glynn Clements <glynn@gclements.plus.com>