[GRASS5] [bug #3023] (grass) r.fillnulls doesn't work

this bug's URL: http://intevation.de/rt/webrt?serial_num=3023
-------------------------------------------------------------------------

Subject: r.fillnulls doesn't work

Platform: GNU/Linux/i386
grass obtained from: Mirror of Trento site
grass binary for platform: Compiled from Sources
GRASS Version: grass-6.0.cvs_src_snapshot_2005_02_12

I'm reffering here to a single 3" SRTM tile loaded using r.in.srtm.

r.fillnulls works very long and uses so much swap that mouse stops answering, temporarily, after several minutes.

Several error infos are printed (see below) and finally r.fillnulls fails to produce anything - in spite of claiming to have "Filled raster map is: N51E016_flnull".

Details:

r.fillnulls input=N51E016 output=N51E016_flnull tension=40. smooth=0.1
Using tmp file r_fillnulls_5276
Locating and isolating NULL areas...
r.buffer: 00000000 - illegal distance specification

Description:
Creates a raster map layer showing buffer zones surrounding cells that contain non-NULL category values.

Usage:
r.buffer [-qz] input=value output=name distances=value[,value,...]
   [units=name]

Flags:
  -q Run quietly
  -z Ignore zero (0) data cells instead of NULL cells

Parameters:
      input Name of input map
     output Name of output map
  distances Distance zone(s)
      units Units of distance
              options: meters,kilometers,feet,miles,nautmiles
              default: meters
invalid map: r_fillnulls_5276.buf
Syntax error
Creating interpolation points...

GRASS_INFO_WARNING(5297,1): Default driver / database set to:
GRASS_INFO_WARNING(5297,1): driver: dbf
GRASS_INFO_WARNING(5297,1): database: $GISDBASE/$LOCATION_NAME/$MAPSET/dbf/

/usr/local/grass-6.0.cvs/scripts/r.fillnulls: line 128: 5297 Broken pipe r.to.vect input=$GIS_OPT_input output=vecttmp_fillnulls_$$ feature=point

Cannot open old vector vecttmp_fillnulls_5276@PERMANENT on level 2
GRASS_INFO_END(5441,1)

Interpolating points

/usr/local/grass-6.0.cvs/scripts/r.fillnulls: line 138: [: -lt: unary operator expected

Note: Below warnings may be ignored.

/usr/local/grass-6.0.cvs/scripts/r.fillnulls: line 166: [: -ge: unary operator expected
Using no segmentation for interpolation as not needed...

GRASS_INFO_WARNING(5449,1): coor files of vector 'vecttmp_fillnulls_5276@PERMANENT' is larger than it should be (39860630 bytes excess).

You first need to run v.build on vector file <vecttmp_fillnulls_5276>

Authors: original version - H.Mitasova, L.Mitas, I. Kosinovsky, D.P. Gerdes
See manual pages for reference and publications
------------------------------------
Note: Above warnings may be ignored.

Patching fill data into NULL areas...
r.patch - rfillnulls_r_fillnulls_5276 not found

GRASS_INFO_WARNING(5453,1): Table 'vecttmp_fillnulls_5276' linked to vector did not exist.

Filled raster map is: N51E016_flnull

Done.

-------------------------------------------- Managed by Request Tracker

this bug's URL: http://intevation.de/rt/webrt?serial_num=3023
---------------------------------------------------------------------

Subject: r.fillnulls doesn't work

(works fine for me on 6.0beta2)

GRASS Version: grass-6.0.cvs_src_snapshot_2005_02_12

I'm reffering here to a single 3" SRTM tile loaded using r.in.srtm.

r.fillnulls works very long and uses so much swap that mouse stops
answering, temporarily, after several minutes.

at which point in the script? Here it takes a couple minutes to run,
long part is duing the v.surf.rst step but no runaway memory issues.
run "top" in another window to watch what is going on?

What is your region set to?
try 'g.region rast=(map)' first?

Several error infos are printed (see below) and finally r.fillnulls
fails to produce anything - in spite of claiming to have "Filled
raster map is: N51E016_flnull".

Details:

r.fillnulls input=N51E016 output=N51E016_flnull tension=40. smooth=0.1

Using tmp file r_fillnulls_5276
Locating and isolating NULL areas...
r.buffer: 00000000 - illegal distance specification

what is the output of "g.region -gm" ??

I've just added some more error checking in CVS, can you update and try
again? maybe uncomment "echo RES=.." line?
(It won't fix your problem but will stop the script if this step fails.)

- In the script we should temporarily zoom to the raster limits with
g.region first?

- I wonder if we should look at using a bigger r.buffer/r.grow area to
get the slope etc at the edges of the hole a bit more correct?

Hamish

From: "Hamish" <hamish_nospam@yahoo.com>
Subject: Re: [GRASS5] [bug #3023] (grass) r.fillnulls doesn't work

this bug's URL: http://intevation.de/rt/webrt?serial_num=3023
---------------------------------------------------------------------

Subject: r.fillnulls doesn't work

(works fine for me on 6.0beta2)

GRASS Version: grass-6.0.cvs_src_snapshot_2005_02_12

I'm reffering here to a single 3" SRTM tile loaded using r.in.srtm.

r.fillnulls works very long and uses so much swap that mouse stops
answering, temporarily, after several minutes.

at which point in the script? Here it takes a couple minutes to run,
long part is duing the v.surf.rst step but no runaway memory issues.
run "top" in another window to watch what is going on?

What is your region set to?
try 'g.region rast=(map)' first?

I did it of course:

GRASS 6.0.cvs:~ > g.region rast=N51E016 -ap
projection: 3 (Latitude-Longitude)
zone: 0
datum: wgs84
ellipsoid: wgs84
north: 52:00:01.5N
south: 50:59:58.5N
west: 15:59:58.5E
east: 17:00:01.5E
nsres: 0:00:03
ewres: 0:00:03
rows: 1201
cols: 1201

Regarding the memory runaway:
- at 20% r.fillnulls starts using swap.
- at 93% the swap is full and mouse hardly responses until done; this is the longest part.
- it doesn't run that long as I thought - 6 minutes alltogether

My system: RAM 512 MB, SWAP 2.0 GB, plenty of free disk space, Athlon XP 1.6, Mandrake 10.1, kernel 2.6.8.

Several error infos are printed (see below) and finally r.fillnulls
fails to produce anything - in spite of claiming to have "Filled
raster map is: N51E016_flnull".

Details:

r.fillnulls input=N51E016 output=N51E016_flnull tension=40. smooth=0.1

Using tmp file r_fillnulls_5276
Locating and isolating NULL areas...
r.buffer: 00000000 - illegal distance specification

what is the output of "g.region -gm" ??

GRASS 6.0.cvs:~ > g.region -g
n=52.00041667
s=50.99958333
w=15.99958333
e=17.00041667
nsres=0.00083333
ewres=0.00083333

What is yours?

I've just added some more error checking in CVS, can you update and try
again? > maybe uncomment "echo RES=.." line?
(It won't fix your problem but will stop the script if this step fails.)

I can't access CVS at home, my expensive, 2kB/s connection doesn't let me to. But I'll grab the next CVS snapshot at my Uni.

- In the script we should temporarily zoom to the raster limits with
g.region first?

Not really a good idea I guess. Anytime it should be up to the user what region he wants to patch.

- I wonder if we should look at using a bigger r.buffer/r.grow area to
get the slope etc at the edges of the hole a bit more correct?

Sounds reasonable to me.

On thing more: my bug report was based r.fillnuls in GUI mode. After running it from the command line I see there are two more errors. Let me quote them all now:

1.
invalid map: r_fillnulls_5276.buf
Syntax error

2. (the "new" one)
dbmi: Protocol error (invalid table/column name or unsupported column type)

3. (the second "new" one)
ERROR: Cannot insert new row: insert into vecttmp_fillnulls_4559 values
       (1376152, 208, '')

4.
ERROR: Cannot open old vector vecttmp_fillnulls_4559@PERMANENT on level 2

5.
WARNING: coor files of vector 'vecttmp_fillnulls_4559@PERMANENT' is larger
         than it should be (39908422 bytes excess).
ERROR: You first need to run v.build on vector file
       <vecttmp_fillnulls_4559>

6.
r.patch - rfillnulls_r_fillnulls_4559 not found
WARNING: Table 'vecttmp_fillnulls_4559' linked to vector did not exist.

(BTW: This is badly annoying that the eventuall error information is different in GUI and comamnd line.)

After after zooming into a small region...

GRASS 6.0.cvs:~ > g.region -m
n=51.5375
s=51.485
w=16.17833333
e=16.2425
nsres=92.71504003
ewres=57.85284109

...only the error no 1. persists:

invalid map: r_fillnulls_5974.buf
Syntax error

And the r.fillnulls manages to output the filled DEM properly (I guess) in spite of it.

Maciek

>> this bug's URL: http://intevation.de/rt/webrt?serial_num=3023
>---------------------------------------------------------------------
>> > Subject: r.fillnulls doesn't work
>

..

>> I'm reffering here to a single 3" SRTM tile loaded using r.in.srtm.
>>
>> r.fillnulls works very long and uses so much swap that mouse stops
>> answering, temporarily, after several minutes.

...?

> What is your region set to?
> try 'g.region rast=(map)' first?

I did it of course:

GRASS 6.0.cvs:~ > g.region rast=N51E016 -ap
projection: 3 (Latitude-Longitude)
zone: 0
datum: wgs84
ellipsoid: wgs84
north: 52:00:01.5N
south: 50:59:58.5N
west: 15:59:58.5E
east: 17:00:01.5E
nsres: 0:00:03
ewres: 0:00:03
rows: 1201
cols: 1201

looks ok.

> what is the output of "g.region -gm" ??

GRASS 6.0.cvs:~ > g.region -g
n=52.00041667
s=50.99958333
w=15.99958333
e=17.00041667
nsres=0.00083333
ewres=0.00083333

that looks ok too. The buffer distance is supposed to be nsres+ewres.
This was fixed to work with floating point resolutions (i.e. lat/lon)
before 6beta2 was released.

> I've just added some more error checking in CVS, can you update and
> try again? maybe uncomment "echo RES=.." line?
> (It won't fix your problem but will stop the script if this step
> fails.)

I can't access CVS at home, my expensive, 2kB/s connection doesn't let
me to. But I'll grab the next CVS snapshot at my Uni.

It is possible from CVS snapshot to get just one file:

cd grass6/scripts/r.fillnulls
cvs -z3 update -dP

that will just update the files in that directory (and any
subdirectories), very little bandwidth.

or from the CVS web interface you can grab the files:

http://freegis.org/cgi-bin/viewcvs.cgi/*checkout*/grass6/scripts/r.in.srtm/r.in.srtm?rev=HEAD&content-type=text/plain

http://freegis.org/cgi-bin/viewcvs.cgi/*checkout*/grass6/scripts/r.fillnulls/r.fillnulls?rev=HEAD&content-type=text/plain

then run "make" from within the modules' directory. (+ install?)

both scripts have new bugfixes and extra checks. can you test them?

> - In the script we should temporarily zoom to the raster limits with
> g.region first?

Not really a good idea I guess. Anytime it should be up to the user
what region he wants to patch.

ok.

On thing more: my bug report was based r.fillnuls in GUI mode. After
running it from the command line I see there are two more errors. Let
me quote them all now:

.. pretty much everything will be broken after the r.buffer step fails.
Please try again with the two updated scripts.

Hamish