[GRASS-dev] RE: [GRASS-user] problems with r.cost

GRASS GIS GRASS GIS developer
1

The large file support option doesn't help with all grass routines
unfortunately.

The following has to be in the Make file for the specific module

ifneq ($(USE_LARGEFILES),)
        EXTRA_CFLAGS = -D_FILE_OFFSET_BITS=64
Endif

It is not in the r.cost Make file, which means I think (remember I'm not a C
programmer) that compiling with large file support won't help.

The nice thing, in principle, about the r.terracost extension is that it can
handle REAAALLY big surfaces. It does some kind of breaking the surface up
into smaller pieces, doing the cost calcs and then patches them back
together, somehow dealing with the issue of going across different pieces.
And also in principle it can be run on a cluster. I keep repeating 'in
principle' because I have never actually done this myself.

Regards, Jerry

-----Original Message-----
From: Paul Haverkamp [mailto:pjhav@ucdavis.edu]
Sent: Wednesday, May 23, 2007 1:51 PM
To: Jerry Nelson; 'Hamish'
Cc: grassuser@grass.itc.it
Subject: RE: [GRASS-user] problems with r.cost

Thanks Jerry,

I'll look into this. I realized that I'm running on a 64 bit machine, so
the memory really shouldn't be an issue. So I'll check out the recompile.

thanks,

paul

One thing to try if you haven't already is to recompile grass with large
file support included. I don't know why that's not done automatically but
apparently it isn't.

Another, if region size is really the issue and you are a C programmer or
know one, is to modify the make file of a grass add-on called
r.terracost.
It was written as a CS project at Bowdoin at a time when grass was at
version 5.3. With the move to version 6, the make process changed
slightly.
The r.terracost files are available at
http://www.bowdoin.edu/~ltoma/research.html

Or you could get grass5.3 and run r.terracost with it.

Regards, Jerry

-----Original Message-----
From: grassuser-bounces@grass.itc.it
[mailto:grassuser-bounces@grass.itc.it]
On Behalf Of Paul Haverkamp
Sent: Wednesday, May 23, 2007 12:45 PM
To: Hamish
Cc: grassuser@grass.itc.it
Subject: Re: [GRASS-user] problems with r.cost

Hi Hamish,

Thanks for getting back to me.

I am using 6.2.1, and g.region -p gives the info:

projection: 1 (UTM)
zone: 10
datum: nad83
ellipsoid: grs80
north: 4269999
south: 4180119
west: 598614
east: 651978
nsres: 3
ewres: 3
rows: 29960
cols: 17788
cells: 532928480

I'm trying to run with the resolution of 3m, but when I change it to 30m,
there isn't a problem. Using much smaller regions doesn't seem to have
this problem. So is the file just too large at too small of a
resolution?
I'll try to run Spearfish and let you know.

thanks again,

paul

> Paul Haverkamp wrote:
> >
> > i'm a very new user to grass and am using r.cost in grass6.2. i'm
> > trying to run a cost analysis on my raster and having some
difficulty.
>
> which version of 6.2? 6.2.0 or 6.2.1?
>
> how big is your region? (g.region -p) Does it happen if you make the
> region smaller or adjust the resolution?
>
> can you reproduce it using the Spearfish demo dataset? if so, can you
> post instructions with that in the grass bug tracker?
>
>
> thanks,
> Hamish
>

_______________________________________________
grassuser mailing list
grassuser@grass.itc.it
http://grass.itc.it/mailman/listinfo/grassuser

2

Jerry Nelson wrote:

The large file support option doesn't help with all grass routines
unfortunately.

The following has to be in the Make file for the specific module

ifneq ($(USE_LARGEFILES),)
        EXTRA_CFLAGS = -D_FILE_OFFSET_BITS=64
Endif

It is not in the r.cost Make file, which means I think (remember I'm not a C
programmer) that compiling with large file support won't help.

For r.cost, adding the above to lib/segment/Makefile would probably be
of more use.

I've been through that code recently, and I'm reasonably sure that
it's safe for LFS.

--
Glynn Clements <glynn@gclements.plus.com>

3

On May 23, 2007, at 2:04 PM, Jerry Nelson wrote:

The large file support option doesn't help with all grass routines
unfortunately.

The following has to be in the Make file for the specific module

ifneq ($(USE_LARGEFILES),)
        EXTRA_CFLAGS = -D_FILE_OFFSET_BITS=64
Endif

It is not in the r.cost Make file, which means I think (remember I'm not a C
programmer) that compiling with large file support won't help.

The need for FILE_OFFSET_BITS and its many variations should be discovered by configure, and whatever is needed by a platform is set in config.h, so this setting here is redundant and possibly meaningless on some platforms.

(OSX doesn't need any of the macros set, and supports large files by default.)

-----
William Kyngesburye <kyngchaos*at*kyngchaos*dot*com>
http://www.kyngchaos.com/

"I ache, therefore I am. Or in my case - I am, therefore I ache."

- Marvin

4

William Kyngesburye wrote:

> The large file support option doesn't help with all grass routines
> unfortunately.
>
> The following has to be in the Make file for the specific module
>
> ifneq ($(USE_LARGEFILES),)
> EXTRA_CFLAGS = -D_FILE_OFFSET_BITS=64
> Endif
>
> It is not in the r.cost Make file, which means I think (remember
> I'm not a C
> programmer) that compiling with large file support won't help.

The need for FILE_OFFSET_BITS and its many variations should be
discovered by configure, and whatever is needed by a platform is set
in config.h, so this setting here is redundant and possibly
meaningless on some platforms.

The settings in config.h are conditionalised upon USE_LARGEFILES,
which is never defined automatically.

If you want LFS, you have to add the relevant flags in the
corresponding Makefile. But first you need to ensure that the code is
actually safe for LFS (i.e. it doesn't use ANSI stdio functions or
store offsets in variables smaller than an off_t) if you don't want
corrupted files.

(OSX doesn't need any of the macros set, and supports large files by
default.)

Unfortunately, much of GRASS doesn't support LFS (especially, but not
exclusively, on platforms where sizeof(long) < sizeof(off_t)), so
enabling it globally is dangerous. That's why you normally have to
specifically enable it with _FILE_OFFSET_BITS etc.

On platforms with a 64-bit "long", you only have to worry about "bad"
code (storing or computing offsets using "int"); even so, GRASS isn't
entirely free from bad code.

--
Glynn Clements <glynn@gclements.plus.com>

5

On May 23, 2007, at 10:09 PM, Glynn Clements wrote:

William Kyngesburye wrote:

The need for FILE_OFFSET_BITS and its many variations should be
discovered by configure, and whatever is needed by a platform is set
in config.h, so this setting here is redundant and possibly
meaningless on some platforms.

The settings in config.h are conditionalised upon USE_LARGEFILES,
which is never defined automatically.

If you want LFS, you have to add the relevant flags in the
corresponding Makefile. But first you need to ensure that the code is
actually safe for LFS (i.e. it doesn't use ANSI stdio functions or
store offsets in variables smaller than an off_t) if you don't want
corrupted files.

(OSX doesn't need any of the macros set, and supports large files by
default.)

Unfortunately, much of GRASS doesn't support LFS (especially, but not
exclusively, on platforms where sizeof(long) < sizeof(off_t)), so
enabling it globally is dangerous. That's why you normally have to
specifically enable it with _FILE_OFFSET_BITS etc.

On platforms with a 64-bit "long", you only have to worry about "bad"
code (storing or computing offsets using "int"); even so, GRASS isn't
entirely free from bad code.

Hm, maybe I've been lucky. I used to manually add _FILE_OFFSET_BITS=64 to my CFLAGS, but stopped doing that a while back. I've been working with >4GB rasters successfully, but I haven't used the more specialized commands like r.cost. I haven't used --enable-largefile thinking it was enabled if the platform supported it. hmmm.

-----
William Kyngesburye <kyngchaos*at*kyngchaos*dot*com>
http://www.kyngchaos.com/

[Trillian] What are you supposed to do WITH a maniacally depressed robot?

[Marvin] You think you have problems? What are you supposed to do if you ARE a maniacally depressed robot? No, don't try and answer, I'm 50,000 times more intelligent than you and even I don't know the answer...

- HitchHiker's Guide to the Galaxy