In Grass 6.3 on Mac OSX 10.5 I try to contour the following raster file (from r.info):
Type of Map: raster Number of Categories: 255
Data Type: CELL
Rows: 38002
Columns: 50276
Total Cells: 1910588552
Projection: UTM (zone 40)
N: 7949005.04157202 S: 7816000 Res: 3.49994847
E: 523000 W: 347033.98326284 Res: 3.50000033
Range of data: min = 0 max = 255
with the region set to the raster file (from g.region -p):
The file size on disk is 1.22 GB. I have 8 GB RAM, whereof 5929 MB free when I execute r.contour on the command line. r.contour fails with the following message after reading in ca. 25% of the data (reproduceable):
GRASS 6.3.0 (NovZem(old)):~ > r.contour --overwrite input=NovZem_S_highrres_red@enhanced output=NovZem_shoreline levels=1
Reading data:
r.contour(13162) malloc: *** mmap(size=405504) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
ERROR: G_malloc: out of memory
This happens while 3020 MB memory are still free! On the other hand it means that r.contour used ca. 3 GB of memory for load ca. 300 MB of data from the disk, i.e. the whole 1.22 GB dataset would require ca. 12 GB of free memory.
Is this behaviour expected?
Best wishes,
Henning
--
Henning Lorenz
Research scientist (Arctic tectonics)
The Swedish Deep Drilling Program (http://www.sddp.se; Scientific coordinator)
Uppsala University
Department of Earth Sciences
Villavägen 16
752 36 Uppsala
Sweden
phone: +46 (0)18 471 23 24
mobile: +46 (0)733 622 655
fax: +46 (0)18 50 11 10
e-mail: henning.lorenz@geo.uu.se http://www.geofys.uu.se/?q=view/home/63
In Grass 6.3 on Mac OSX 10.5 I try to contour the following raster
file (from r.info):
Type of Map: raster Number of Categories: 255
Data Type: CELL
Rows: 38002
Columns: 50276
Total Cells: 1910588552
Projection: UTM (zone 40)
N: 7949005.04157202 S: 7816000 Res: 3.49994847
E: 523000 W: 347033.98326284 Res: 3.50000033
Range of data: min = 0 max = 255
with the region set to the raster file (from g.region -p):
The file size on disk is 1.22 GB. I have 8 GB RAM, whereof 5929 MB
free when I execute r.contour on the command line. r.contour fails
with the following message after reading in ca. 25% of the data
(reproduceable):
GRASS 6.3.0 (NovZem(old)):~ > r.contour --overwrite
input=NovZem_S_highrres_red@enhanced output=NovZem_shoreline levels=1
Reading data:
r.contour(13162) malloc: *** mmap(size=405504) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
ERROR: G_malloc: out of memory
This happens while 3020 MB memory are still free! On the other hand it
means that r.contour used ca. 3 GB of memory for load ca. 300 MB of
data from the disk, i.e. the whole 1.22 GB dataset would require ca.
12 GB of free memory.
Is this behaviour expected?
Yes.
You have ~2G (2^31) cells. Each one is stored internally as a 64-bit
floating-point value[1], meaning that it would require ~16GiB of
memory.
Try reducing the region resolution.
Note that the file in question is compressed, and only uses 1 byte per
cell when decompressed.
[1] which is probably overkill; a 32-bit value would likely suffice.
However, that might still be problematic in this particular case,
depending upon how much memory is required for other purposes.
On Sep 22, 2008, at 8:13 AM, Glynn Clements wrote:
The file size on disk is 1.22 GB. I have 8 GB RAM, whereof 5929 MB
free when I execute r.contour on the command line. r.contour fails
with the following message after reading in ca. 25% of the data
(reproduceable):
GRASS 6.3.0 (NovZem(old)):~ > r.contour --overwrite
input=NovZem_S_highrres_red@enhanced output=NovZem_shoreline levels=1
Reading data:
r.contour(13162) malloc: *** mmap(size=405504) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
ERROR: G_malloc: out of memory
This happens while 3020 MB memory are still free! On the other hand it
means that r.contour used ca. 3 GB of memory for load ca. 300 MB of
data from the disk, i.e. the whole 1.22 GB dataset would require ca.
12 GB of free memory.
Is this behaviour expected?
Yes.
You have ~2G (2^31) cells. Each one is stored internally as a 64-bit
floating-point value[1], meaning that it would require ~16GiB of
memory.
Try reducing the region resolution.
Note that the file in question is compressed, and only uses 1 byte per
cell when decompressed.
[1] which is probably overkill; a 32-bit value would likely suffice.
However, that might still be problematic in this particular case,
depending upon how much memory is required for other purposes.
Also note that in a 32bit OSX build (what my binaries are) you will have memory problems at the 2GB (maybe 4GB?) mark. It doesn't matter if you're running on a 64bit Mac.
I don't build 64bit GRASS because there are some myterious problems in the vector library that haven't been worked out. Damn, I keep forgetting to submit a bug report.