[GRASS-dev] Max Memory usage limit

GRASS GIS GRASS GIS developer
,
1

Hi all,

Just out of curiosity, why is there an upper limit to the memory we can use on operations like r.in.gdal?

with kind regards,
Panos

2

On Mon, Jan 11, 2016 at 8:04 PM, Panagiotis Mavrogiorgos
<pmav99@gmail.com> wrote:

Hi all,

Just out of curiosity, why is there an upper limit to the memory we can use
on operations like r.in.gdal?

The question is too generic, so I'll refer to r.in.gdal only:
To my knowledge there is a GDAL imposed CACHE limit

https://trac.osgeo.org/grass/browser/grass/trunk/raster/r.in.gdal/main.c#L271

271 /* default GDAL memory cache size appears to be only 40 MiB,
slowing down r.in.gdal */
272 if (parm.memory->answer && *parm.memory->answer) {
273 /* TODO: GDALGetCacheMax() overflows at 2GiB, implement
use of GDALSetCacheMax64() */
274 GDALSetCacheMax(atol(parm.memory->answer) * 1024 * 1024);

but concerning the data you can import I am not aware of any limit.

Maybe this page is of interest to you:
https://grasswiki.osgeo.org/wiki/Large_raster_data_processing

What is exactly your problem you encountered?

Markus

3

On Tue, Jan 12, 2016 at 10:20 AM, Markus Neteler <neteler@osgeo.org> wrote:

On Mon, Jan 11, 2016 at 8:04 PM, Panagiotis Mavrogiorgos
<pmav99@gmail.com> wrote:
> Hi all,
>
> Just out of curiosity, why is there an upper limit to the memory we can
use
> on operations like r.in.gdal?

The question is too generic, so I'll refer to r.in.gdal only:
To my knowledge there is a GDAL imposed CACHE limit

https://trac.osgeo.org/grass/browser/grass/trunk/raster/r.in.gdal/main.c#L271

271 /* default GDAL memory cache size appears to be only 40 MiB,
slowing down r.in.gdal */
272 if (parm.memory->answer && *parm.memory->answer) {
273 /* TODO: GDALGetCacheMax() overflows at 2GiB, implement
use of GDALSetCacheMax64() */
274 GDALSetCacheMax(atol(parm.memory->answer) * 1024 * 1024);

but concerning the data you can import I am not aware of any limit.

Maybe this page is of interest to you:
https://grasswiki.osgeo.org/wiki/Large_raster_data_processing

What is exactly your problem you encountered?

Thank you Marcus,

That was really helpful. I don't have a particular problem. I just found
strange that the limit was so low on x64 architectures.

with kind regards,
Panos

4

On Jan 13, 2016 9:28 AM, “Panagiotis Mavrogiorgos” <pmav99@gmail.com> wrote:
…

That was really helpful. I don’t have a particular problem. I just found strange that the limit was so low on x64 architectures.

Which limit do you refer to?

Best
Markus