* Roberta Fagandini <robifagandini@gmail.com> [2018-06-04 16:14:16 +0200]:
Hi all!
Thank you, Moritz and Nikos, for your suggestions!
2018-06-04 11:54 GMT+02:00 Nikos Alexandris <nik@nikosalexandris.net>:
Ciao Roberta,
great work!
Moritz got me faster, as I intended to chime in, with similar
suggestions.
So +1 for:
- a "real" module, as Moritz described it
Done! I added the folder to my github repository but I have some trouble
compiling the module with g.extension:
g.extension extension=i.sentinel.mask operation=add url=
https://github.com/RobiFag/GRASS_clouds_and_shadows
error message:
g.extension extension=i.sentinel.mask operation=add url=
https://github.com/RobiFag/GRASS_clouds_and_shadows
Fetching <i.sentinel.mask> from <
https://github.com/RobiFag/GRASS_clouds_and_shadows/archive/master.zip> (be
patient)...
Compiling...
make[1]: *** No rule to make target `/tmp/grass7-roberta-701
4/tmpe7QEso/i.sentinel.mask/scripts/i.sentinel.mask', needed
by `script'. Stop.
/bin/sh: 1: cannot create /usr/lib/grass74/error.log:
Permission denied
make: *** [i.sentinel.mask] Error 2
ERROR: Compilation failed, sorry. Please check above error messages.
any suggestion?
Maybe irrelevant, but perhaps it is best to name the script as
`i.sentinel.mask.py`?
- for using the metadata file, and parsing it -- my experience with
Landsat and other high-resolution satellite products, showed it's best to
trust the individual metadata that accompany an acquisition.
Parameters may be very specific to one single acquisition.
Also, a module that is designed to be "dynamic" is more
future-proof and easy to maintain and update.
For example, reading band names from the meta-data, as opposed to more
"static" designs, i.e. expecting specific hardcoded options, such as
band names, or their order.
I have already thought about parsing the metadatafile to retrieve the input
bands but at the moment the module requires atmospherically corrected bands
and if users perform the atmospheric correction on their own I don't have
any control on naming. Moreover, the module needs to know which raster map
corresponds to the blue band, to the green band and so on in order to apply
the rules for clouds and shadows detection. Therefore I need users specify
each band but maybe I can manage this issue requiring a specific suffix in
the bands' names (e.g. blabla_blue, blabla_green, etc.). In this way, users
have to provide only the prefix (blabla) while the module search for all
raster maps with the specified prefix and at the same time it is able to
recognize each band.
Anyway, the aim of the next coding period is to create a python script that
wraps in a single GRASS module the download and import phase
(i.sentinel.download and i.sentinel.import), the atmospheric correction
using i.atcorr and the cloud and shadow detection procedure. In particular,
for what concerns the atmospheric correction, I'd like to implement an
iterative procedure that executes i.atcorr for all bands of the input image
changing accordingly the requested input parameters. This part should
simplify the management of input maps.
One thing here: massive processing workflows like to have simple
processing building blocks. One Input -> one Process -> one Output. In
"your" case:
- Input: name of/request one band and associated
metadata, and optionally other ancillary data.
- Process: download, import, removal of atmospheric effects to estimate
surface reflectance
- Output: surface reflectance ("corrected" for atmospheric effects) of
the requested input band
This building block can run in one processing units/node. Making use of many
processing units/nodes, this block can run in parallel for different
input bands.
I suggest to build the module in a way that it runs for one band or as many
bands as the user requires for his workflow. I did this in
https://gitlab.com/NikosAlexandris/i.fusion.hpf/blob/master/i.fusion.hpf.py.
I don't suggest this script as an example. Yet I think the concept "for
one or many" is useful and wanted.
As a sidenote:
I think that we don't use the term "correction"
correctly. An image acquired by a satellite-borne sensor is not "wrong"
to need a correction. Rather, we try to retouch the digital form of the
acquisition, in order to estimate the energy that was actually reflected
by the surface, before the signal travels through the atmosphere.
I studied remote sensing with Thomas Lillesand's; Ralph W.
Kiefer's book "Remote Sensing and Image Interpretation". The book
"correctly" touches this very question.
Maybe it's only a perspective, and others have another say on this.
- PEP8 rules and keeping module options in separate lines to increase
legibility
In line with the suggestions on style, I would stress out to fully
spell out the names of functions, options, descriptions of flags,
iterables and iterators. It will make it easier for the next reader of
your code.
Yes, you are both right!! I have to clean up the code from the style point
of view and it will be one of the tasks of this week and especially of the
last days before the GSoC Evaluation.
Finally, it's a good idea to write even mini-tests, while you are
progressing, for the small functions that eventually develop to keep the
full scrip compact, functional and more legible.
Writing a test, for your module, in the end, will be much easier.
Sorry Nikos but I don't really understand what you mean by
"mini-tests"..are you referring to the examples in the manual page?
For a function f(x) that takes x as an input and gives you y as output,
test, for example, whether the output value 'y' is within the expected
range. Test your single functions, inside the script. Use assertions, for
example, etc. If you have 10 functions in your script, and have written
a test for each, you will save your time later on when you will revisit
the code to debug or modify. Spend as much time as you can in testing.
Again, this is what I try to stick to. And, lastly, I am not a core
developer, nor a mentor. I am, however, interested in seeing this module
coming alive. So, apologies to the mentors that I am chimingi in.
[rest deleted]
Grazie, Nikos
