[GRASS5] r.mapcalc and NULL question

GRASS GIS GRASS GIS developer
1

Hi there,

maybe a r.mapcalc/NULL FAQ, but I don't get it...

Assume you have a streams map (streams contain values,
other cells=NULL).

Now I want to add some values *everywhere* using r.mapcalc, but
it is only added in those cells not containing a NULL value.

Any ideas how to overcome this problem?

Thanks in advance

Markus

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

2

Rich,

(cc to the list for some more explanations)

On Mon, Jan 15, 2001 at 09:35:13AM -0800, Rich Shepard wrote:

On Mon, 15 Jan 2001, Markus Neteler wrote:

> Assume you have a streams map (streams contain values, other cells=NULL).
>
> Now I want to add some values *everywhere* using r.mapcalc, but it is only
> added in those cells not containing a NULL value.
>
> Any ideas how to overcome this problem?

Markus,

  Can you replace the NULL with a finite value such as -99 (for example)? If
so, then you can set up a case to replace -99 with whatever value you want.

  Alternatively, what I used to do in Idrisi is to build a new map with all
cells containing zeros. Then I add my streams map to the zero map and now
all non-stream cells have the value of zero. That removes the nulls and
leaves the original maps intact.

HTH,

Rich,

thanks for your quick response. Of course I could do that, but I
am searching for a general solution as I am script programming (with
loops).
So the users don't want convert their maps qithin every loop... :slight_smile:
Generally r.null would do the job, but that's too time consuming for my
purposes.

The NULLs I would like to keep for overlayed displaying (this wouldn't
work with ZEROs).

I am writing a script for my students to stepwise flood landscapes
with graphical output on the monitor for each step.

Hope there is a built-in r.mapcalc solution which I have overseen...

Markus

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

3

Markus,

Maybe I am misunderstanding but r.mapcalc does have null value support
..

eg.
mapcalc> if (isnull(streams), #newval, streams)

The isnull() mapcalc function looks for all null vals in raster. I
believe you can also use null() to set a value to null.

A mapcalc / NULL FAQ might be a good idea.
--
Bob Covill

Tekmap Consulting
P.O. Box 2016
Fall River, N.S.
B2T 1K6
Canada

E-Mail: bcovill@tekmap.ns.ca
Phone: 902-860-1496
Fax: 902-860-1498

4

On Mon, 15 Jan 2001, Markus Neteler wrote:

Hi there,

maybe a r.mapcalc/NULL FAQ, but I don't get it...

Assume you have a streams map (streams contain values,
other cells=NULL).

Now I want to add some values *everywhere* using r.mapcalc, but
it is only added in those cells not containing a NULL value.

Markus,

I guess you probably want the NULL-valued cells to be treated like zeros.
To add maps A and B (where B contains NULLs) to get a map C you can use a
construction like

C=A+if(isnull(B),0,B)

I found a need for this when I used Grass 5 to run the Leicestershire
tutorial, which was written for Grass 4.x.

There's another NULL-related behavior that I find rather annoying. I
guess that the numerical value of NULL is actually -129. I constructed a
raster map of water-level changes from a ground water model and found a
band of nulls scattered across the map, wherever the surface passed -129
feet.

Roger Miller
Lee Wilson and Associates

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

5

On Mon, Jan 15, 2001 at 12:41:58PM -0700, Roger S. Miller wrote:

On Mon, 15 Jan 2001, Markus Neteler wrote:

> Hi there,
>
> maybe a r.mapcalc/NULL FAQ, but I don't get it...
>
> Assume you have a streams map (streams contain values,
> other cells=NULL).
>
> Now I want to add some values *everywhere* using r.mapcalc, but
> it is only added in those cells not containing a NULL value.

Markus,

I guess you probably want the NULL-valued cells to be treated like zeros.
To add maps A and B (where B contains NULLs) to get a map C you can use a
construction like

C=A+if(isnull(B),0,B)

I found a need for this when I used Grass 5 to run the Leicestershire
tutorial, which was written for Grass 4.x.

There's another NULL-related behavior that I find rather annoying. I
guess that the numerical value of NULL is actually -129. I constructed a
raster map of water-level changes from a ground water model and found a
band of nulls scattered across the map, wherever the surface passed -129
feet.

No, the NULL value depends on the CELL type. Think this is/was a bug in
r.mapcalc specifically (If I recall, it has been corrected). Generally
the embedded NULL values (when the file is read via G_* funcs) are

CELL : MAX_INT
FCELL : NaN (float version)
DCELL : Nan (double version)

However, applications should always use the G_is_null_value() family of
functions. The CELL Null value is problematic if used incorrectly
because you can silently get overflow/underflow errors. NaN used in any
context should always produce NaN (NaN = NaN - NaN) -- at least, that's
how it's supposed to work :wink:

--
Eric G. Miller <egm2@jps.net>

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

6

On Mon, Jan 15, 2001 at 12:40:21PM -0800, Eric G . Miller wrote:

On Mon, Jan 15, 2001 at 12:41:58PM -0700, Roger S. Miller wrote:
>
> On Mon, 15 Jan 2001, Markus Neteler wrote:
>
> > Hi there,
> >
> > maybe a r.mapcalc/NULL FAQ, but I don't get it...
> >
> > Assume you have a streams map (streams contain values,
> > other cells=NULL).
> >
> > Now I want to add some values *everywhere* using r.mapcalc, but
> > it is only added in those cells not containing a NULL value.

Perhaps I am not very clear... I will try to find a better example.
I got the impression, that value replacing is o.k. (replace NULL by
something else), but only if you use the isnull() function.

Yes, just now I got an example:

- take a map containing NULL holes (say, it is "map1")
- take a second map without NULLS (say, it is "map2")

Now I want to fill the NULLs in map one with values of map2:

Option 1:
r.mapcalc new="if(isnull(map1), map2 + 1000., map1)" # make a silly addition
r.mapcalc new="if(isnull(map1), map2, map1)" # replace value
-> both are o.k.

But (Option 2):

r.mapcalc new="if(map1, map2 + 1000., map1)"
r.mapcalc new="if((map1), map2, map1)"
-> NULLs are not replaced.

Is it a bug or a feature? From you kind responses I was at least aware
what's going on here :slight_smile:

Generally I consider this as a bug because you can't always know if
a map contains NULLs. If only a few pixels would be nulls you may
use option 2 and don't realize that the result is not want you want.
I feel this behaviour is somewhat dangerous. The isnull() is useful,
but if you have to apply it on default in if-clauses, it's a bit annoying.

Other comments?

> Markus,
>
> I guess you probably want the NULL-valued cells to be treated like zeros.
> To add maps A and B (where B contains NULLs) to get a map C you can use a
> construction like
>
> C=A+if(isnull(B),0,B)
>
> I found a need for this when I used Grass 5 to run the Leicestershire
> tutorial, which was written for Grass 4.x.

Thanks for the hint! From that I could construct above example.

> There's another NULL-related behavior that I find rather annoying. I
> guess that the numerical value of NULL is actually -129. I constructed a
> raster map of water-level changes from a ground water model and found a
> band of nulls scattered across the map, wherever the surface passed -129
> feet.

No, the NULL value depends on the CELL type. Think this is/was a bug in
r.mapcalc specifically (If I recall, it has been corrected).

Yes, this bug is definitly fixed for FreeBSD, Solaris and Linux.
Which platform are you using?

From changelog:

2000-08-20 16:10 markus
        * null_val.c: fix for Linux on -129 NULL bug

src/libes/gis/null_val.c (here the fix has to go).

Try
r.mapcalc test=-129
CREATING SUPPORT FILES FOR test
range: -129 -129

If this is not the result, you may not use Linux or have a very old GRASS 5
version.

Generally
the embedded NULL values (when the file is read via G_* funcs) are

CELL : MAX_INT
FCELL : NaN (float version)
DCELL : Nan (double version)

However, applications should always use the G_is_null_value() family of
functions. The CELL Null value is problematic if used incorrectly
because you can silently get overflow/underflow errors. NaN used in any
context should always produce NaN (NaN = NaN - NaN) -- at least, that's
how it's supposed to work :wink:

Markus

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

7

On Mon, 15 Jan 2001, Markus Neteler wrote:

Generally I consider this as a bug because you can't always know if
a map contains NULLs. If only a few pixels would be nulls you may
use option 2 and don't realize that the result is not want you want.
I feel this behaviour is somewhat dangerous. The isnull() is useful,
but if you have to apply it on default in if-clauses, it's a bit annoying.

Other comments?

I first felt that it was a bug, but the best alternative I could required
a default value to be used in the place of null. Zero is the most obvious
choice for a default value, but I could think of plenty of cases where
that wasn't a good idea. Requiring the user to provide a value for null
also seemed like a bad idea. So I decided it must be a feature.

For what it's worth, Golden Software's Surfer package (and Topo before
that) has for years used a value of 1.70141e+038 as a null value -- they
use it for "blanking". Their grid math calculations treat blanks pretty
much the same way that r.mapcalc treats null; any operation on a blank
cell returns a blank cell.

> No, the NULL value depends on the CELL type. Think this is/was a bug in
> r.mapcalc specifically (If I recall, it has been corrected).
Yes, this bug is definitly fixed for FreeBSD, Solaris and Linux.
Which platform are you using?

Thanks, I see what happened.

I am running Linux, but I was running grass beta7 at the time. That was
in November, just before I updated to beta10. I didn't start reading this
list until after the code was changed, so I missed any announcement or
discussion.

I guess this is evidence that problems do get fixed :slight_smile:

Roger Miller
Lee Wilson and Associates

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

8

On Mon, Jan 15, 2001 at 03:07:48PM -0700, Roger S. Miller wrote:

On Mon, 15 Jan 2001, Markus Neteler wrote:

> Generally I consider this as a bug because you can't always know if
> a map contains NULLs. If only a few pixels would be nulls you may
> use option 2 and don't realize that the result is not want you want.
> I feel this behaviour is somewhat dangerous. The isnull() is useful,
> but if you have to apply it on default in if-clauses, it's a bit annoying.
>
> Other comments?

I first felt that it was a bug, but the best alternative I could required
a default value to be used in the place of null. Zero is the most obvious
choice for a default value, but I could think of plenty of cases where
that wasn't a good idea. Requiring the user to provide a value for null
also seemed like a bad idea. So I decided it must be a feature.

For what it's worth, Golden Software's Surfer package (and Topo before
that) has for years used a value of 1.70141e+038 as a null value -- they
use it for "blanking". Their grid math calculations treat blanks pretty
much the same way that r.mapcalc treats null; any operation on a blank
cell returns a blank cell.

I tend to agree with Roger here. There doesn't seem to be an obvious
way to handle the scenario, so trying to outsmart the user may create
more problems than solutions. Maybe we need to get the word out better
than any math performed with NULL values should always result in a NULL
value. This is the very definition of NULL as an "undefined/unknown value".

--
Eric G. Miller <egm2@jps.net>

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

9

Hi Markus

Markus Neteler wrote:

Generally I consider this as a bug because you can't always know if
a map contains NULLs. If only a few pixels would be nulls you may
use option 2 and don't realize that the result is not want you want.
I feel this behaviour is somewhat dangerous. The isnull() is useful,
but if you have to apply it on default in if-clauses, it's a bit
annoying.

Other comments?

I agree with the others, it is not a bug. Perhaps r.mapcalc should print
a warning if it tries to perform arithmetic on null values. At least
then, the user will be notified of the problem.

--
Sincerely,

Jazzman (a.k.a. Justin Hickey) e-mail: jhickey@hpcc.nectec.or.th
High Performance Computing Center
National Electronics and Computer Technology Center (NECTEC)
Bangkok, Thailand

People who think they know everything are very irritating to those
of us who do. ---Anonymous

Jazz and Trek Rule!!!

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

10

On Tue, Jan 16, 2001 at 03:44:39PM +0700, Justin Hickey wrote:

Hi Markus

Markus Neteler wrote:
> Generally I consider this as a bug because you can't always know if
> a map contains NULLs. If only a few pixels would be nulls you may
> use option 2 and don't realize that the result is not want you want.
> I feel this behaviour is somewhat dangerous. The isnull() is useful,
> but if you have to apply it on default in if-clauses, it's a bit
> annoying.
>
> Other comments?

I agree with the others, it is not a bug. Perhaps r.mapcalc should print
a warning if it tries to perform arithmetic on null values. At least
then, the user will be notified of the problem.

O.k. I am convinced :slight_smile: The current behaviour will be the best.
But printing a "WARNING: Map %s contains NULLs" might be helpful.

Then we can add a comment to html/html/r.mapcalc.html that arithmetics
on NULL cells lead to NULL cells unless isnull() is used.

Meanwhile I found such a workaround for my script...

Markus

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

11

On Mon, Jan 15, 2001 at 02:44:47PM -0800, Eric G . Miller wrote:

On Mon, Jan 15, 2001 at 03:07:48PM -0700, Roger S. Miller wrote:
>
> On Mon, 15 Jan 2001, Markus Neteler wrote:
>
>
> > Generally I consider this as a bug because you can't always know if
> > a map contains NULLs. If only a few pixels would be nulls you may
> > use option 2 and don't realize that the result is not want you want.
> > I feel this behaviour is somewhat dangerous. The isnull() is useful,
> > but if you have to apply it on default in if-clauses, it's a bit annoying.
> >
> > Other comments?
>
> I first felt that it was a bug, but the best alternative I could required
> a default value to be used in the place of null. Zero is the most obvious
> choice for a default value, but I could think of plenty of cases where
> that wasn't a good idea. Requiring the user to provide a value for null
> also seemed like a bad idea. So I decided it must be a feature.
>
> For what it's worth, Golden Software's Surfer package (and Topo before
> that) has for years used a value of 1.70141e+038 as a null value -- they
> use it for "blanking". Their grid math calculations treat blanks pretty
> much the same way that r.mapcalc treats null; any operation on a blank
> cell returns a blank cell.

I tend to agree with Roger here. There doesn't seem to be an obvious
way to handle the scenario, so trying to outsmart the user may create
more problems than solutions. Maybe we need to get the word out better
than any math performed with NULL values should always result in a NULL
value. This is the very definition of NULL as an "undefined/unknown value".

Hi Eric, hi all,

I have added such a comment to html/html/r.mapcalc.html
Will be uploaded to the web site this night from CVS by
the web server's cronjob.

BTW: My flooding script is working by using the isnull() function.

Thanks for this discussion!

Markus

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'