[GRASS-dev] pyGRASS and mapcalc's pixel modifiers

GRASS GIS GRASS GIS developer
1

Hello programmers.

Consider a Python class with methods that derive a somewhat complex mapcalc
expression which contains the following pixel modifiers:

['A[-1, -1]', 'A[-1, 0]', 'A[-1, 1]', 'A[0, -1]', 'A[0, 0]', 'A[0, 1]',
'A[1, -1]', 'A[1, 0]', 'A[1, 1]']

These modifiers built-up a moving window to compute some values (eg. mean, median and other in-between values) and finally end up in a polynomial like:

-9.674 + 0.653 * SomeValue + 9.087 * SomeValue^2

As far as my understanding goes, I can't check for the validity of the
in-between computed values with mapcalc's powers. For example, check if the
average value as derived from the above defined pixel modifiers, lies within a
specific range. I have build helper functions, as part of the Python class,
which could do this. Then again, I don't have the powers of mapcalc to process
the data.

I wonder how I should approach this, excluding rewriting my program in C (which
I don't speak easily). If I go step-by-step, that is:

1) compute a map based with the mean of neighboring pixels (see pix-modifiers above)
2) use pyGRASS' RasterRow, for example, to be able to apply some helper function and check or do something
3) compute another, required, in-between map
4) use pyGRASS again
5) etc.

up until to reach the final equation of interest. This sounds really messy. How would you approach this (excluding an
implementation in C)?

Nikos

2

On 06/05/15 12:25, Nikos Alexandris wrote:

Hello programmers.

Consider a Python class with methods that derive a somewhat complex mapcalc
expression which contains the following pixel modifiers:

['A[-1, -1]', 'A[-1, 0]', 'A[-1, 1]', 'A[0, -1]', 'A[0, 0]', 'A[0, 1]',
'A[1, -1]', 'A[1, 0]', 'A[1, 1]']

These modifiers built-up a moving window to compute some values (eg. mean, median and other in-between values) and finally end up in a polynomial like:

-9.674 + 0.653 * SomeValue + 9.087 * SomeValue^2

As far as my understanding goes, I can't check for the validity of the
in-between computed values with mapcalc's powers. For example, check if the
average value as derived from the above defined pixel modifiers, lies within a
specific range. I have build helper functions, as part of the Python class,
which could do this. Then again, I don't have the powers of mapcalc to process
the data.

I wonder how I should approach this, excluding rewriting my program in C (which
I don't speak easily). If I go step-by-step, that is:

1) compute a map based with the mean of neighboring pixels (see pix-modifiers above)
2) use pyGRASS' RasterRow, for example, to be able to apply some helper function and check or do something
3) compute another, required, in-between map
4) use pyGRASS again
5) etc.

up until to reach the final equation of interest. This sounds really messy. How would you approach this (excluding an
implementation in C)?

Have a look at the eval function of r.mapcalc (check the specific section of the man page). It allows you to create intermediate results which you can then use in further processing.

Moritz

3

* Moritz Lennert <mlennert@club.worldonline.be> [2015-05-06 12:59:16 +0200]:

On 06/05/15 12:25, Nikos Alexandris wrote:
> Hello programmers.
>
> Consider a Python class with methods that derive a somewhat complex mapcalc
> expression which contains the following pixel modifiers:
>
> ['A[-1, -1]', 'A[-1, 0]', 'A[-1, 1]', 'A[0, -1]', 'A[0, 0]', 'A[0, 1]',
> 'A[1, -1]', 'A[1, 0]', 'A[1, 1]']
>
>
> These modifiers built-up a moving window to compute some values (eg. mean, median and other in-between values) and finally end up in a polynomial like:
>
> -9.674 + 0.653 * SomeValue + 9.087 * SomeValue^2
>
>
> As far as my understanding goes, I can't check for the validity of the
> in-between computed values with mapcalc's powers. For example, check if the
> average value as derived from the above defined pixel modifiers, lies within a
> specific range. I have build helper functions, as part of the Python class,
> which could do this. Then again, I don't have the powers of mapcalc to process
> the data.
>
> I wonder how I should approach this, excluding rewriting my program in C (which
> I don't speak easily). If I go step-by-step, that is:
>
> 1) compute a map based with the mean of neighboring pixels (see pix-modifiers above)
> 2) use pyGRASS' RasterRow, for example, to be able to apply some helper function and check or do something
> 3) compute another, required, in-between map
> 4) use pyGRASS again
> 5) etc.
>
> up until to reach the final equation of interest. This sounds really messy. How would you approach this (excluding an
> implementation in C)?

Have a look at the eval function of r.mapcalc (check the specific
section of the man page). It allows you to create intermediate results
which you can then use in further processing.

Thank you Moritz. Already in use. And quite happy about it. eval, anyhow,
isn't an answer to my question.

A simple example of what would satisfy the current need, would be the
application of a helper function, like the following, on the fly. That is
while reading or computing pixel values.

def check_cwv(cwv):
    """
    Check whether a column water vapor value lies within a "valid" range.
    """
    if cwv < 0.0 or cwv > 6.3:
        raise ValueError('The column water vapor estimation is out of the '
                         'expected ranfe [0.0, 6.3]')
    else:
        return True

Thanks, Nikos

4

On 06/05/15 14:42, Nikos Alexandris wrote:

* Moritz Lennert <mlennert@club.worldonline.be> [2015-05-06 12:59:16 +0200]:

On 06/05/15 12:25, Nikos Alexandris wrote:

Hello programmers.

Consider a Python class with methods that derive a somewhat complex mapcalc
expression which contains the following pixel modifiers:

['A[-1, -1]', 'A[-1, 0]', 'A[-1, 1]', 'A[0, -1]', 'A[0, 0]', 'A[0, 1]',
'A[1, -1]', 'A[1, 0]', 'A[1, 1]']

These modifiers built-up a moving window to compute some values (eg. mean, median and other in-between values) and finally end up in a polynomial like:

-9.674 + 0.653 * SomeValue + 9.087 * SomeValue^2

As far as my understanding goes, I can't check for the validity of the
in-between computed values with mapcalc's powers. For example, check if the
average value as derived from the above defined pixel modifiers, lies within a
specific range. I have build helper functions, as part of the Python class,
which could do this. Then again, I don't have the powers of mapcalc to process
the data.

I wonder how I should approach this, excluding rewriting my program in C (which
I don't speak easily). If I go step-by-step, that is:

1) compute a map based with the mean of neighboring pixels (see pix-modifiers above)
2) use pyGRASS' RasterRow, for example, to be able to apply some helper function and check or do something
3) compute another, required, in-between map
4) use pyGRASS again
5) etc.

up until to reach the final equation of interest. This sounds really messy. How would you approach this (excluding an
implementation in C)?

Have a look at the eval function of r.mapcalc (check the specific
section of the man page). It allows you to create intermediate results
which you can then use in further processing.

Thank you Moritz. Already in use. And quite happy about it. eval, anyhow,
isn't an answer to my question.

A simple example of what would satisfy the current need, would be the
application of a helper function, like the following, on the fly. That is
while reading or computing pixel values.

def check_cwv(cwv):
     """
     Check whether a column water vapor value lies within a "valid" range.
     """
     if cwv < 0.0 or cwv > 6.3:
         raise ValueError('The column water vapor estimation is out of the '
                          'expected ranfe [0.0, 6.3]')
     else:
         return True

It's clear that you cannot raise an error with such a message in r.mapcalc, but I don't understand why you couldn't do such a test (as long as this test is pixel by pixel) ? Instead of the message you could give those pixels a specific value (e.g. -1 or -9999 or whatever).

But I guess I just don't really understand what you are after...

Moritz

5

Nikos Alexandris wrote:

>>> Consider a Python class with methods that derive a somewhat complex mapcalc
>>> expression which contains the following pixel modifiers:
>>>
>>> ['A[-1, -1]', 'A[-1, 0]', 'A[-1, 1]', 'A[0, -1]', 'A[0, 0]', 'A[0, 1]',
>>> 'A[1, -1]', 'A[1, 0]', 'A[1, 1]']

>>> These modifiers built-up a moving window to compute some values (eg. mean, median and other in-between values) and finally end up in a polynomial like:
>>>
>>> -9.674 + 0.653 * SomeValue + 9.087 * SomeValue^2
>>>
>>> As far as my understanding goes, I can't check for the validity of the
>>> in-between computed values with mapcalc's powers. For example, check if the
>>> average value as derived from the above defined pixel modifiers, lies within a
>>> specific range. I have build helper functions, as part of the Python class,
>>> which could do this. Then again, I don't have the powers of mapcalc to process
>>> the data.
>>>
>>> I wonder how I should approach this, excluding rewriting my program in C (which
>>> I don't speak easily). If I go step-by-step, that is:
>>>
>>> 1) compute a map based with the mean of neighboring pixels (see pix-modifiers above)
>>> 2) use pyGRASS' RasterRow, for example, to be able to apply some helper function and check or do something
>>> 3) compute another, required, in-between map
>>> 4) use pyGRASS again
>>> 5) etc.
>>>
>>> up until to reach the final equation of interest. This sounds really messy. How would you approach this (excluding an
>>> implementation in C)?

Moritz:

>> Have a look at the eval function of r.mapcalc (check the specific
>> section of the man page). It allows you to create intermediate results
>> which you can then use in further processing.

Nikos:

> Thank you Moritz. Already in use. And quite happy about it. eval,
> anyhow, > > isn't an answer to my question. A simple example of what would
> satisfy the current need, would be the > > application of a helper function,
> like the following, on the fly. That is > > while reading or computing pixel
> values.

> def check_cwv(cwv):
> """
> Check whether a column water vapor value lies within a "valid" range.
> """
> if cwv < 0.0 or cwv > 6.3:
> raise ValueError('The column water vapor estimation is out of the '
> 'expected ranfe [0.0, 6.3]')
> else:
> return True

Moritz Lennert:

It's clear that you cannot raise an error with such a message in
r.mapcalc, but I don't understand why you couldn't do such a test (as
long as this test is pixel by pixel) ? Instead of the message you could
give those pixels a specific value (e.g. -1 or -9999 or whatever).

Hmmm? You mean to use if-else conditionals, assign an "out-of-range"
value for pixes which are "faulty"?

That sounds like an idea, but will this make my computations more clean?
Thanks a lot for caring & sharing Moritz.

But I guess I just don't really understand what you are after...

I am after adding checks in various steps of the algorithm I am implementing.
It's just a somewhat complex equation [*]. That's all. Hopefully I'll get
it complete and share results soon.

Nikos

[*] Huazhong Ren, Chen Du, Qiming Qin, Rongyuan Liu,
Jinjie Meng, and Jing Li. "Atmospheric Water Vapor Retrieval
from Landsat 8 and Its Validation." 3045–3048. IEEE, 2014.

6

* Nikos Alexandris <nik@nikosalexandris.net> [2015-05-06 19:26:17 +0300]:
..

I am after adding checks in various steps of the algorithm I am implementing.
It's just a somewhat complex equation [*]. That's all. Hopefully I'll get
it complete and share results soon.

Here is the implementation of [0] for which [*] is a part:
<https://github.com/NikosAlexandris/i.landsat8.swlst&gt;

Nikos

[*] Huazhong Ren, Chen Du, Qiming Qin, Rongyuan Liu,
Jinjie Meng, and Jing Li. "Atmospheric Water Vapor Retrieval
from Landsat 8 and Its Validation." 3045–3048. IEEE, 2014.

[0] [0] Du, Chen; Ren, Huazhong; Qin, Qiming; Meng, Jinjie; Zhao,
Shaohua. 2015. "A Practical Split-Window Algorithm for Estimating Land
Surface Temperature from Landsat 8 Data." Remote Sens. 7, no. 1:
647-665. http://www.mdpi.com/2072-4292/7/1/647/htm\\\#sthash\.ba1pt9hj\.dpuf