[GRASSLIST:1601] Problem reprojecting...

Hi,

With some RTFMing and a bunch of help from Bruce at Baylor, I have managed
to import my DOQQ's into grass using r.in.gdal. I can't figure out how to
use the native colormap of the tiff yet, but that is a small issue.

r.in.tiff still does not handle these images well (bringing my load
average to over 1 for eternity, and never completing the import.)
This is from a cvs checkout done on friday, March 9th.

I now have two locations, one in SPCS and one in UTM.

I need to reproject one mapset in the UTM location to the SPCS location.

If someone has some time and could help me through this, I would
appreciate it.

r.proj keeps telling me that the projection areas don't overlap, which I
believe is not correct.

Thanks,
Matt Berglund

On Mon, Mar 12, 2001 at 02:03:59PM -0500, mberglund wrote:

Hi,

With some RTFMing and a bunch of help from Bruce at Baylor, I have managed
to import my DOQQ's into grass using r.in.gdal. I can't figure out how to
use the native colormap of the tiff yet, but that is a small issue.

?? Is this a color DOQQ? Was it palette based or 24bit? I would've
thunk that r.in.gdal would handle the colormap (if there was one). If
it's multi-band (rgb), set each band to 0 = black, 255 = white. Then,
d.rgb or i.composite... Where'd you get these DOQQ's from? They
weren't by chance exported from Arc/Info grid without the use of a
colormap? I've done that, and you get 16-bit TIFF's that are useless.

r.in.tiff still does not handle these images well (bringing my load
average to over 1 for eternity, and never completing the import.)
This is from a cvs checkout done on friday, March 9th.

Other than the addition of TILE support, no significant work has been
done on how r.in.tiff works for a long time.

I now have two locations, one in SPCS and one in UTM.

I need to reproject one mapset in the UTM location to the SPCS location.

If someone has some time and could help me through this, I would
appreciate it.

r.proj keeps telling me that the projection areas don't overlap, which I
believe is not correct.

Be careful of r.proj! I've filed a bug against it for reading the
entire raster into memory before projecting it. It can cream your
system. i.rectify is an alternative (and smarter about the memory),
though you need tie points.

--
Eric G. Miller <egm2@jps.net>

On Mon, 12 Mar 2001, Eric G. Miller wrote:

Be careful of r.proj! I've filed a bug against it for reading the
entire raster into memory before projecting it. It can cream your
system. i.rectify is an alternative (and smarter about the memory),
though you need tie points.

Not true any longer since grass5.0beta10 or so. r.proj now does not
allocate more memory than necessary and only reads in the overlapping
region of the input map.

r.proj is also "region sensitive", i.e. you have to set your current
location to the 'default region' in order to get a map that covers the
whole location. if you have set your current region to a smaller area and
the input map does not cover all of the location you _may_ get an 'input
map is outside current region' error.

Morten Hulden

On Thu, Mar 15, 2001 at 12:46:10AM +0100, Morten Hulden wrote:

On Mon, 12 Mar 2001, Eric G. Miller wrote:

> Be careful of r.proj! I've filed a bug against it for reading the
> entire raster into memory before projecting it. It can cream your
> system. i.rectify is an alternative (and smarter about the memory),
> though you need tie points.

Not true any longer since grass5.0beta10 or so. r.proj now does not
allocate more memory than necessary and only reads in the overlapping
region of the input map.

So, if I'm projecting a very large raster, it reads the whole thing...

r.proj is also "region sensitive", i.e. you have to set your current
location to the 'default region' in order to get a map that covers the
whole location. if you have set your current region to a smaller area and
the input map does not cover all of the location you _may_ get an 'input
map is outside current region' error.

Well, trying to project a 78MB image, with 64MB RAM and 200 MB swap, and
I had to kill the process just before it killed my system.

... main.c ...

339 G_set_window(&incellhd);
340 fdi = G_open_cell_old(inmap->answer, setname);
341 ibuffer = (FCELL **) readcell(fdi);
342 G_close_cell(fdi);
343 G__switch_env();

... readcell.c ...

21 nrows = G_window_rows();
22
23 ibuffer = (FCELL **) G_malloc(sizeof(FCELL **) * nrows);
24
25 fprintf(stderr,"Allocating memory and reading input map... ");
26 G_percent(0, nrows, 5);
27 for (row = 0; row < nrows; row++){
28 ibuffer[row] = (FCELL *) G_allocate_raster_buf(FCELL_TYPE);
29 if(G_get_raster_row(fdi, ibuffer[row], row, FCELL_TYPE) < 0)
30 G_fatal_error("Error reading input");
31 G_percent(row, nrows - 1, 5);
32 }
33 return(ibuffer);

Looks like it reads the whole thing to me, barring region sensitive
clipping... IMHO, it should only read as many rows as it currently
needs (which would be a max of 3, no?). So the reads should be nested
in with the writes...

--
Eric G. Miller <egm2@jps.net>

On Wed, 14 Mar 2001, Eric G. Miller wrote:

So, if I'm projecting a very large raster, it reads the whole thing...

... main.c ...

339 G_set_window(&incellhd);
340 fdi = G_open_cell_old(inmap->answer, setname);
341 ibuffer = (FCELL **) readcell(fdi);
342 G_close_cell(fdi);
343 G__switch_env();

The above used to be at the begining of main.c ( version <
grass5.0beta10), which was very annoying because r.proj always started by
allocating memory for the whole input map, and did so even if the map was
completely outside the output region. I inserted checks and map trimming
functions that abort the module (if there are no overlapping parts) or
change the incellhd before calling readcell() so only overlapping parts
are read.

Looks like it reads the whole thing to me, barring region sensitive
clipping... IMHO, it should only read as many rows as it currently
needs (which would be a max of 3, no?). So the reads should be nested
in with the writes...

It depends on the size of the output region. But yes, if your output
region is as large as, or larger than your input map, the whole map will
be read into memory.

Of course you are right that this should be done dynamically, but that
means a major rewrite of r.proj, and creates other problems. E.g. it may
be necessary to read in the same part of the map several times due to
special shape and rotation relations between input and output maps, or
due to special interpolation methods.

But for now, if you are low on memory you can always do the projections in
parts, using the region sensitivity of r.proj. That is, define subregions
in the output location and project into each region separately. Only
the part of the input map corresponding to the current region will be
read into memory. Then use r.patch to get one single map.

regards
Morten

I had been following the clustering discussion a few days ago and ran across
this site today:

http://www.mosix.org/

I wonder how useful this kind of clustering framework would be for GRASS
applications. It seems pretty straightforward to set up.
--
Eric Hillmuth
Principal
Digital Meridian
http://www.digitalmeridian.net
Phone: 303-809-8972
Fax: 303-443-9427

On Fri, 6 Apr 2001, Eric Hillmuth wrote:

I had been following the clustering discussion a few days ago and ran across
this site today:

http://www.mosix.org/

I wonder how useful this kind of clustering framework would be for GRASS
applications. It seems pretty straightforward to set up.

Eric,

  It certainly looks impressively easy! Now, if I had the spare cash to set
up a bunch of linux boxes in a rack, I could test it. :slight_smile:

  Worth looking at it detail, I suggest.

Thanks,

Rich

Dr. Richard B. Shepard, President

                       Applied Ecosystem Services, Inc. (TM)
            2404 SW 22nd Street | Troutdale, OR 97060-1247 | U.S.A.
+ 1 503-667-4517 (voice) | + 1 503-667-8863 (fax) | rshepard@appl-ecosys.com