[GRASS-user] How does GRASS do tiled processing?

GRASS GRASS user
1

GRASSers:

    I was curious -- how is tiled processing realized in GRASS GIS? Is there a fixed input tile size (in MB of RAM or # of lines)? Is there some documentation buried on the GRASS site that describes the algorithm? I'm trying to replicate an efficient tiled approach in R -- I was basing it off the ENVI approach (precalculate the input data memory footprint per line of data, read in as many lines as the memory cap allows, process, write those lines, rinse, repeat), but I was curious if GRASS had a different approach.

--j

--

Jonathan A. Greenberg, PhD
Postdoctoral Scholar
Center for Spatial Technologies and Remote Sensing (CSTARS)
University of California, Davis
One Shields Avenue
The Barn, Room 250N
Davis, CA 95616
Cell: 415-794-5043
AIM: jgrn307, MSN: jgrn307@hotmail.com, Gchat: jgrn307

2

Hi Jonathan,

When I need to do tiles processing of grass coupled R, I usually set a list of bounding boxes on R (a list of x1, x2, y1, y2), and then I put it on a for() looping. So, I set a new g.region using n= s= e= and w= parameters using system() function of R (you can do it of other ways). Just after the for() I reset g.region with -d.

but you need to be very careful with your processing, because some of the results will be influenced by the boundary of new sub-regions.

Good luck

milton
brazil=toronto

2009/7/8 Jonathan Greenberg <greenberg@ucdavis.edu>

GRASSers:

I was curious – how is tiled processing realized in GRASS GIS? Is there a fixed input tile size (in MB of RAM or # of lines)? Is there some documentation buried on the GRASS site that describes the algorithm? I’m trying to replicate an efficient tiled approach in R – I was basing it off the ENVI approach (precalculate the input data memory footprint per line of data, read in as many lines as the memory cap allows, process, write those lines, rinse, repeat), but I was curious if GRASS had a different approach.

–j

–

Jonathan A. Greenberg, PhD
Postdoctoral Scholar
Center for Spatial Technologies and Remote Sensing (CSTARS)
University of California, Davis
One Shields Avenue
The Barn, Room 250N
Davis, CA 95616
Cell: 415-794-5043
AIM: jgrn307, MSN: jgrn307@hotmail.com, Gchat: jgrn307

grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

3

Milton:

    Thanks, but I'm more curious in just the generic way GRASS does tiled processing (say, in mapcalc). I assume there is a low-level processing layer GRASS uses (or no?). I'm not doing a direct grass-to-R link, I'm doing the processing completely within R with rgdal, but I'm interested in various solutions to the tiled processing problem.

    That is a helpful suggestion, tho -- I used a similar approach when I "forced" r.sun to do tiled processing with a massive lidar image I had -- I ended up setting overlaps between each tile since its a spatial problem. For pure pixel analyses, of course, there's no need to worry about boundaries...

--j

Milton Cezar Ribeiro wrote:

Hi Jonathan,
When I need to do tiles processing of grass coupled R, I usually set a list of bounding boxes on R (a list of x1, x2, y1, y2), and then I put it on a for() looping. So, I set a new g.region using n= s= e= and w= parameters using system() function of R (you can do it of other ways). Just after the for() I reset g.region with -d.
*but* you need to be very careful with your processing, because some of the results will be influenced by the boundary of new sub-regions.
Good luck
milton
brazil=toronto

2009/7/8 Jonathan Greenberg <greenberg@ucdavis.edu <mailto:greenberg@ucdavis.edu>>

    GRASSers:

      I was curious -- how is tiled processing realized in GRASS GIS?
     Is there a fixed input tile size (in MB of RAM or # of lines)?
     Is there some documentation buried on the GRASS site that
    describes the algorithm? I'm trying to replicate an efficient
    tiled approach in R -- I was basing it off the ENVI approach
    (precalculate the input data memory footprint per line of data,
    read in as many lines as the memory cap allows, process, write
    those lines, rinse, repeat), but I was curious if GRASS had a
    different approach.

    --j

    --

    Jonathan A. Greenberg, PhD
    Postdoctoral Scholar
    Center for Spatial Technologies and Remote Sensing (CSTARS)
    University of California, Davis
    One Shields Avenue
    The Barn, Room 250N
    Davis, CA 95616
    Cell: 415-794-5043
    AIM: jgrn307, MSN: jgrn307@hotmail.com
    <mailto:jgrn307@hotmail.com>, Gchat: jgrn307
    _______________________________________________
    grass-user mailing list
    grass-user@lists.osgeo.org <mailto:grass-user@lists.osgeo.org>
    http://lists.osgeo.org/mailman/listinfo/grass-user

--

Jonathan A. Greenberg, PhD
Postdoctoral Scholar
Center for Spatial Technologies and Remote Sensing (CSTARS)
University of California, Davis
One Shields Avenue
The Barn, Room 250N
Davis, CA 95616
Cell: 415-794-5043
AIM: jgrn307, MSN: jgrn307@hotmail.com, Gchat: jgrn307

4

Jonathan Greenberg wrote:

    Thanks, but I'm more curious in just the generic way GRASS does
tiled processing (say, in mapcalc). I assume there is a low-level
processing layer GRASS uses (or no?).

GRASS' primary raster processing model is row-by-row. Where possible,
the code reads a row, processes it, writes a row. If a neighbourhood
is required, the code typically maintains a scrolling window; it may
use the rowio library for this, or do it itself (r.mapcalc uses
rowio).

Modules which need uncorrelated access either read the entire map into
RAM, or use an uncompressed temporary file which may be a simple 2D
array or may be tiled. If it's tiled, the code may use the segment
library, or do it itself.

--
Glynn Clements <glynn@gclements.plus.com>