[GRASS-dev] Added i.landsat8.swlst in grass-addons

GRASS GIS GRASS GIS developer
1

Dear all,

I've been asked off-line some times for the custom script available at
https://github.com/NikosAlexandris/i.landsat8.swlst -- specifically on how to
get it installed.

I added this now in GRASS Add-Ons since the easiest way, for new users,
to install one, is to use g.extension.

I hope I didn't do anything wrong in adding this. Due to lack of time,
I didn't check if all is well. Please, let me know in case I overlooked
something that might hinder the normal workflow in the addons
repository.

Cheers, Nikos

2

On Wed, Apr 20, 2016 at 9:54 AM, Nikos Alexandris <nik@nikosalexandris.net>
wrote:

I hope I didn't do anything wrong in adding this. Due to lack of time,
I didn't check if all is well. Please, let me know in case I overlooked
something that might hinder the normal workflow in the addons
repository.

It seems that it's fine. The online manual page is compiled and you
Submitting Python in your TODO. Only some pictures are missing in manual
page.

https://grass.osgeo.org/grass70/manuals/addons/i.landsat8.swlst.html

3

Dear Nikos,

I just tested your i.landsat8.swlst module. Cool stuff!

During my experiments I ran into one issue and identified some (minor) room for speed-ups (mainly replacing g.copy with g.rename). I did not do a full code review but made some suggestions in a pull request on github. I tested all changes and they seem to work fine.

Some other things I noticed beyond my changes:

  • For me the manual did not build locally (meaning it was not available in the GUI)

  • Given the fact that the FROM-GLC data has not been updated for a while (if I did not get the link wrong), I would encourage users to provide their own land cover map, possibly reclassified from topographic maps. The only one available at http://data.ess.tsinghua.edu.cn/ for my scene was not only a couple of years old, but also quite clowdy in addition.

  • The k-flag (keep current computational region) is invers to GRASS standards. Maybe better to switch the behavior?

  • The GUI would – for my taste – benefit from splitting the options over different tabs (guisection). Now most options and flags end up in “optional” which makes it a bit confusing for newcomers to see what is input and what output…

Anyway, great tool!

Cheers

Stefan

···

On Wed, Apr 20, 2016 at 9:54 AM, Nikos Alexandris <nik@nikosalexandris.net> wrote:

I hope I didn’t do anything wrong in adding this. Due to lack of time,
I didn’t check if all is well. Please, let me know in case I overlooked
something that might hinder the normal workflow in the addons
repository.

It seems that it’s fine. The online manual page is compiled and you Submitting Python in your TODO. Only some pictures are missing in manual page.

https://grass.osgeo.org/grass70/manuals/addons/i.landsat8.swlst.html

4

* Blumentrath, Stefan <Stefan.Blumentrath@nina.no> [2016-05-06 13:09:31 +0000]:

Dear Nikos,

I just tested your i.landsat8.swlst module. Cool stuff!

Thank you (so) very much for testing!

During my experiments I ran into one issue and identified some (minor)
room for speed-ups (mainly replacing g.copy with g.rename).

Nice :slight_smile:

I did not
do a full code review but made some suggestions in a pull request on
github. I tested all changes and they seem to work fine.

I tested your changes, it works. Merged. Some questions I (might) have (for my own
learning though), I'll put as comments in github.

Some other things I noticed beyond my changes:

- For me the manual did not build locally (meaning it was not available in the GUI)

It builds for me, not error-free though (missing images).

- Given the fact that the FROM-GLC data has not been updated for
a while (if I did not get the link wrong), I would encourage users to
provide their own land cover map, possibly reclassified from
topographic maps. The only one available at
http://data.ess.tsinghua.edu.cn/ for my scene was not only a couple of
years old, but also quite clowdy in addition.

Yes. Have you already done this for your area of interest?

- The k-flag (keep current computational region) is invers to GRASS standards. Maybe better to switch the behavior?

Not sure. It's kind of related to the #1 mistake everyone does, forget
setting the right region extent. So, I'd thought of letting the module
operate on the whole scene.

Do we have a reference for this ("GRASS standards") or you
mean the "common" behaviour of most modules?

But, in any case, I am all open to any "good" change(s)! If you have changed this
locally, please include it in a pull request.

- The GUI would – for my taste – benefit from splitting the
options over different tabs (guisection). Now most options and flags
end up in “optional” which makes it a bit confusing for newcomers to
see what is input and what output...

Absolutely! This is simply work not done. I tried hard, really, to be clean
with the flags. I have to re-read the script, I almost don't remember a
thing at the moment! I think I let it be like that because I did not
resolve on what and how should be required, and how this would affect
other options. Need to re-read.

Thank you Stefan, really awesome you took the time to read and apply
corrections. It's most rewarding to learn collaboratively!

Kindest regards, Nikos

5

On Sat, May 7, 2016 at 5:14 PM, Nikos Alexandris <nik@nikosalexandris.net>
wrote:

- The k-flag (keep current computational region) is invers to GRASS

standards. Maybe better to switch the behavior?

Not sure. It's kind of related to the #1 mistake everyone does, forget
setting the right region extent. So, I'd thought of letting the module
operate on the whole scene.

Then really all GRASS modules should be changed.

Do we have a reference for this ("GRASS standards") or you
mean the "common" behaviour of most modules?

No. Perhaps we need another Submitting page on Trac or perhaps just adding
a best practice to the main Submitting page.

This particular thing should be also somewhere in the user documentation
(which obviously should be followed when developing a new module). It seems
that g.region man page uses outdated terminology and doesn't do a good job
on describing this behavior. Anyone, please contribute a better text.

Vaclav

6

On Mon, May 9, 2016 at 2:10 AM, Vaclav Petras <wenzeslaus@gmail.com> wrote:

On Sat, May 7, 2016 at 5:14 PM, Nikos Alexandris <nik@nikosalexandris.net>

- The k-flag (keep current computational region) is invers to
GRASS standards. Maybe better to switch the behavior?

Not sure. It's kind of related to the #1 mistake everyone does, forget
setting the right region extent. So, I'd thought of letting the module
operate on the whole scene.

Then really all GRASS modules should be changed.

Yeah, probably not :slight_smile:

Do we have a reference for this ("GRASS standards") or you
mean the "common" behaviour of most modules?

No. Perhaps we need another Submitting page on Trac or perhaps just adding a
best practice to the main Submitting page.

Here you go:
https://trac.osgeo.org/grass/wiki/Submitting/General#Modulebehaviour:computationalregionsettings

This particular thing should be also somewhere in the user documentation
(which obviously should be followed when developing a new module). It seems
that g.region man page uses outdated terminology and doesn't do a good job
on describing this behavior. Anyone, please contribute a better text.

It could be taken from here:

https://grasswiki.osgeo.org/wiki/Computational_region

Markus

7

Hi Nikos,

There are no real benefits from removing the kelvin_to_celsius function except for that code is a few lines shorter and the function is no longer used (plus that it is unlikely that you will be using it in the module in future). But the same applies probably to the "copy" helper function and the other stuff I just commented out could be removed too I guess, but I am not educated as a programmer. So others are much more qualified to comment on coding styles...

However, now I also had a closer look at the output of i.landsat8.swlst from a methodological point of view. Here it seems that - at least in my case of the city of Oslo - the difference between land-cover-corrected-LST and not land-cover-corrected-LST (I used land cover type 70 as a constant which has an average emissivity value) is almost insignificant (~ +- 1 degree celsius). The max/min differences were between -20 and +20 degrees Celsius, but these values appeared only at the boundary of the LST maps. Within the scenes differences were never > +-5 degree and very rarely > +-2.
Given the amount of data and computation required to incorporate land cover effects, you might consider to make a land cover map optional and just use a constant in the map calculator expression if no landcover map is provided...?

Another issue I came across it that currently, a possibly existing user mask will be simply overwritten, while it might be more friendly to backup the user mask before and restore it after the module finished... However, in order to simplify the module a bit, an option might be to move the masking part out of the module, and probably write something like this: https://grass.osgeo.org/grass70/manuals/i.modis.qc.html for Landsat, if that does not exist yet(?). Seems you thought about it already?
Based on Markus tutorial (http://courses.neteler.org/processing-landsat8-data-in-grass-gis-7/#Applying_the_Landsat_8_Quality_Assessment_(QA)_Band) and the info here http://landsat.usgs.gov/qualityband.php that should be a quite doable job. If no landsat bitpattern module exists I might be able to volunteer...
The idea would be something like this: extract raster values from QA band, loop over those values, compare the bit pattern representation of the integer values with acceptable bit patterns defined by the user, add the result of the comparison to r.reclass rules, and finally reclassify the QA band...

But all in all the results look pretty good, which you could see here: http://urban.nina.no/layers/geonode%3Alc81980182015183lgn00_lst and here: http://urban.nina.no/layers/geonode%3Alc81980182015183lgn00_lst\_const\_lc if our GeoNode worked as it should...

Kind regards,
Stefan

-----Original Message-----
From: Nikos Alexandris [mailto:nikos.alexandris@unige.ch]
Sent: 9. mai 2016 16:31
To: Vaclav Petras <wenzeslaus@gmail.com>
Cc: Nikos Alexandris <nik@nikosalexandris.net>; Blumentrath, Stefan <Stefan.Blumentrath@nina.no>; grass-dev@lists.osgeo.org
Subject: Re: [GRASS-dev] Added i.landsat8.swlst in grass-addons

* Vaclav Petras <wenzeslaus@gmail.com> [2016-05-08 20:10:46 -0400]:

On Sat, May 7, 2016 at 5:14 PM, Nikos Alexandris
<nik@nikosalexandris.net>
wrote:

- The k-flag (keep current computational region) is invers to GRASS

standards. Maybe better to switch the behavior?

Not sure. It's kind of related to the #1 mistake everyone does,
forget setting the right region extent. So, I'd thought of letting
the module operate on the whole scene.

Then really all GRASS modules should be changed.

:slight_smile: -- Let's change'm all!

I am joking, of course. Will alter the behaviour in the module.

Do we have a reference for this ("GRASS standards") or you mean the
"common" behaviour of most modules?

No. Perhaps we need another Submitting page on Trac or perhaps just
adding a best practice to the main Submitting page.

Yep!

Cheers, Nikos

[..]

8

Hi again,

I just pushed a new addon ("i.landsat8.qc") to addons svn.
i.landsat8.qc reclassifies Landsat8 QA band according to a user defined filter for bit pattern representing unacceptable quality pixels. It implements basically the procedure mentioned below...

Any feedback is welcome...

Kind regards,
Stefan

-----Original Message-----
From: grass-dev [mailto:grass-dev-bounces@lists.osgeo.org] On Behalf Of Blumentrath, Stefan
Sent: 10. mai 2016 09:57
To: Nikos Alexandris <nikos.alexandris@unige.ch>; Vaclav Petras <wenzeslaus@gmail.com>
Cc: grass-dev@lists.osgeo.org
Subject: Re: [GRASS-dev] Added i.landsat8.swlst in grass-addons

Another issue I came across it that currently, a possibly existing user mask will be simply overwritten, while it might be more friendly to backup the user mask before and restore it after the module finished... However, in order to simplify the module a bit, an option might be to move the masking part out of the module, and probably write something like this: https://grass.osgeo.org/grass70/manuals/i.modis.qc.html for Landsat, if that does not exist yet(?). Seems you thought about it already?
Based on Markus tutorial (http://courses.neteler.org/processing-landsat8-data-in-grass-gis-7/#Applying_the_Landsat_8_Quality_Assessment_(QA)_Band) and the info here http://landsat.usgs.gov/qualityband.php that should be a quite doable job. If no landsat bitpattern module exists I might be able to volunteer...
The idea would be something like this: extract raster values from QA band, loop over those values, compare the bit pattern representation of the integer values with acceptable bit patterns defined by the user, add the result of the comparison to r.reclass rules, and finally reclassify the QA band...