[GRASS-user] NVIZ can't allocate enough memory

GRASS GRASS user
1

Hi list,

I’m having some trouble using NVIZ. It’s worked before using similar data, so I’m not quite sure what the problem is. My goal is to overlay aerial photos over a digital surface model, but I always get errors that GRASS can’t allocate enough memory. It doesn’t matter how small I make the region. The same problem occurs when I try to do that without the aerial photos, or if I try to take e.g. the red band from the photo and use it as height without putting a photo on top of it. Here’s the error:

(Mon Jun 11 10:54:34 2012)
nviz elevation=dom_complete@PERMANENT
ERROR: G_malloc: unable to allocate 18446744071974792324 bytes at gsds.c:575
(Mon Jun 11 10:54:34 2012) Command finished (0 sec)

Does anybody know what could be the problem? If I’m interpreting the bytes correctly, that’d be about 10^10 gigabytes, which several orders of magnitude larger than the rasters involved. Thanks!

Best,
Daniel

2

On Mon, Jun 11, 2012 at 11:10 AM, Daniel Lee <lee@isi-solutions.org> wrote:

Hi list,

I'm having some trouble using NVIZ. It's worked before using similar data,
so I'm not quite sure what the problem is. My goal is to overlay aerial
photos over a digital surface model, but I always get errors that GRASS
can't allocate enough memory. It doesn't matter how small I make the region.

Please post
g.region -p

The same problem occurs when I try to do that without the aerial photos, or
if I try to take e.g. the red band from the photo and use it as height
without putting a photo on top of it. Here's the error:

(Mon Jun 11 10:54:34
2012)
nviz
elevation=dom_complete@PERMANENT
ERROR: G_malloc: unable to allocate 18446744071974792324 bytes at gsds.c:575
(Mon Jun 11 10:54:34 2012) Command finished (0
sec)

To me it looks like a g.region user error.

Markus

3

Hi Markus,

Here’s the output of g.region:

GRASS 6.4.2 (EPSG31467_GK3):~ > g.region -p
projection: 99 (Transverse Mercator)
zone: 0
datum: potsdam
ellipsoid: bessel
north: 5867905.9439733
south: 5851898.90994448
west: 3430043.32300553
east: 3440047.55082599
nsres: 0.50020418
ewres: 0.50018638
rows: 32001
cols: 20001
cells: 640052001

Looks alright to me. I set it exactly to the raster in question by using g.region rast=dom_complete.

Any ideas? Thanks! :slight_smile:

Daniel

–

B.Sc. Daniel Lee
Geschäftsführung für Forschung und Entwicklung
ISIS - International Solar Information Solutions GbR
Vertreten durch: Daniel Lee, Nepomuk Reinhard und Nils Räder

Softwarecenter 3
35037 Marburg
Festnetz: +49 6421 379 6256
Mobil: +49 176 6127 7269
E-Mail: Lee@isi-solutions.org
Web: http://www.isi-solutions.org

2012/6/11 Markus Neteler <neteler@osgeo.org>

On Mon, Jun 11, 2012 at 11:10 AM, Daniel Lee <lee@isi-solutions.org> wrote:

Hi list,

I’m having some trouble using NVIZ. It’s worked before using similar data,
so I’m not quite sure what the problem is. My goal is to overlay aerial
photos over a digital surface model, but I always get errors that GRASS
can’t allocate enough memory. It doesn’t matter how small I make the region.

Please post
g.region -p

The same problem occurs when I try to do that without the aerial photos, or
if I try to take e.g. the red band from the photo and use it as height
without putting a photo on top of it. Here’s the error:

(Mon Jun 11 10:54:34
2012)
nviz
elevation=dom_complete@PERMANENT
ERROR: G_malloc: unable to allocate 18446744071974792324 bytes at gsds.c:575
(Mon Jun 11 10:54:34 2012) Command finished (0
sec)

To me it looks like a g.region user error.

Markus

4

On Mon, 11 Jun 2012, Daniel Lee wrote:

ERROR: G_malloc: unable to allocate 18446744071974792324 bytes at gsds.c:575

Daniel,

   That's more than 1,717,9869,182 Gigabytes of memory. It appears the
resolution of your aerial photograph is much too high.

Rich

--
Richard B. Shepard, Ph.D. | Integrity - Credibility - Innovation
Applied Ecosystem Services, Inc. | Helping Ensure Our Clients' Futures
<http://www.appl-ecosys.com> Voice: 503-667-4517 Fax: 503-667-8863

5

I agree with Rich that your resolution might be too fine. I would either change the resolution to something coarser i. e. : g.region res=5
or make your region smaller. Definitely 640 million cells seems like a lot of cells to me.

Cheers.

On Mon, Jun 11, 2012 at 9:03 AM, Rich Shepard <rshepard@appl-ecosys.com> wrote:

On Mon, 11 Jun 2012, Daniel Lee wrote:

ERROR: G_malloc: unable to allocate 18446744071974792324 bytes at gsds.c:575

Daniel,

That’s more than 1,717,9869,182 Gigabytes of memory. It appears the
resolution of your aerial photograph is much too high.

Rich

–
Richard B. Shepard, Ph.D. | Integrity - Credibility - Innovation
Applied Ecosystem Services, Inc. | Helping Ensure Our Clients’ Futures
<http://www.appl-ecosys.com> Voice: 503-667-4517 Fax: 503-667-8863

grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

6

Thanks for the tips! I cropped the region and it worked like a charm :slight_smile:

Best,
Daniel

7

On Mon, Jun 11, 2012 at 11:10 AM, Daniel Lee <lee@isi-solutions.org> wrote:

Hi list,

I'm having some trouble using NVIZ.

...

nviz
elevation=dom_complete@PERMANENT
ERROR: G_malloc: unable to allocate 18446744071974792324 bytes at gsds.c:575

@devs: I would suggest to enrich this error message and print out
the current rows/cols along with a suggestion to verify the current region
settings.

Markus

8

Daniel Lee wrote:

ERROR: G_malloc: unable to allocate 18446744071974792324 bytes at gsds.c:575

18446744071974792324 = 0xffffffff9899ac84

Does anybody know what could be the problem? If I'm interpreting the bytes
correctly, that'd be about 10^10 gigabytes, which several orders of
magnitude larger than the rasters involved. Thanks!

A calculation has overflowed the range of a signed 32-bit integer
resulting in a negative value. This value has then been converted to a
signed 64-bit integer, and then to an unsigned 64-bit integer.

Here's the output of g.region:

rows: 32001
cols: 20001
cells: 640052001

At 4 bytes per cell, 640052001 cells = 2560208004 bytes.

2560208004 = 0x9899ac84

The maximum value representable by a 32-bit integer is 0x7fffffff =
2147483647.

Using an unsigned integer would eliminate the problem in this
particular case, but the region wouldn't need to be much larger in
order for that to be insufficient (specifically, 32001 x 33553 would
overflow).

The correct solution is to use "size_t" rather than "int". E.g. for
this specific case:

--- gsds.c (revision 52015)
+++ gsds.c (working copy)
@@ -481,7 +481,8 @@
int gsds_alloc_typbuff(int id, int *dims, int ndims, int type)
{
     dataset *ds;
- int i, siz = 1;
+ int i;
+ size_t siz = 1;

     if ((ds = get_dataset(id))) {
   /*

More generally, it's important that the conversion comes before the
mulitply; e.g.:

  size_t cells = (size_t) rows * cols;

will work but:

  size_t cells = rows * cols;

won't; the calculation will be perfomed using "int", overflow, then
the overflowed result will be promoted.

However: the number of instances of this particular issue in the GRASS
source code is probably somewhere between "dozens" and "hundreds", and
there's no systematic way to identify them. Running test cases with a
region of >= 2^32 cells (or even >= 2^29 cells, i.e. >= 2^31 bytes at
4 bytes per cell) would find a lot of them, but it requires a 64-bit
system with plenty of RAM, and it's going to be slow.

--
Glynn Clements <glynn@gclements.plus.com>

9

On Tue, Jun 12, 2012 at 5:23 AM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Daniel Lee wrote:

ERROR: G_malloc: unable to allocate 18446744071974792324 bytes at gsds.c:575

18446744071974792324 = 0xffffffff9899ac84

Does anybody know what could be the problem? If I'm interpreting the bytes
correctly, that'd be about 10^10 gigabytes, which several orders of
magnitude larger than the rasters involved. Thanks!

A calculation has overflowed the range of a signed 32-bit integer
resulting in a negative value. This value has then been converted to a
signed 64-bit integer, and then to an unsigned 64-bit integer.

Here's the output of g.region:

rows: 32001
cols: 20001
cells: 640052001

At 4 bytes per cell, 640052001 cells = 2560208004 bytes.

2560208004 = 0x9899ac84

The maximum value representable by a 32-bit integer is 0x7fffffff =
2147483647.

Using an unsigned integer would eliminate the problem in this
particular case, but the region wouldn't need to be much larger in
order for that to be insufficient (specifically, 32001 x 33553 would
overflow).

The correct solution is to use "size_t" rather than "int". E.g. for
this specific case:

--- gsds.c (revision 52015)
+++ gsds.c (working copy)
@@ -481,7 +481,8 @@
int gsds_alloc_typbuff(int id, int *dims, int ndims, int type)
{
dataset *ds;
- int i, siz = 1;
+ int i;
+ size_t siz = 1;

if \(\(ds = get\_dataset\(id\)\)\) \{
   /\*

More generally, it's important that the conversion comes before the
mulitply; e.g.:

   size\_t cells = \(size\_t\) rows \* cols;

will work but:

   size\_t cells = rows \* cols;

won't; the calculation will be perfomed using "int", overflow, then
the overflowed result will be promoted.

However: the number of instances of this particular issue in the GRASS
source code is probably somewhere between "dozens" and "hundreds", and
there's no systematic way to identify them. Running test cases with a
region of >= 2^32 cells (or even >= 2^29 cells, i.e. >= 2^31 bytes at
4 bytes per cell) would find a lot of them, but it requires a 64-bit
system with plenty of RAM, and it's going to be slow.

I have started modifying some libs, but still need to fix (checks for)
return values.

Markus M

--
Glynn Clements <glynn@gclements.plus.com>
_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user