[GRASS-dev] seg fault of i.atcorr on quickbird

ychemin · November 27, 2016, 6:47pm

Hi,

trying to run i.atcorr on an image of Colombo.

i.atcorr --overwrite input=radxs.1 parameters=6s_b1.txt output=refxs.1

param file:

17 - geometrical conditions=Quickbird
2 16 5.40 79.860 6.920 - month day hh.ddd longitude latitude ("hh.ddd" is in decimal hours GMT)
1 - atmospheric mode=tropical
2 - aerosols model=maritime
15 - visibility [km] (aerosol model concentration)
-0.001 - mean target elevation above sea level [km] (here 1m asl)
-1000 - sensor height (here, sensor on board a satellite)
111 - Blue band of Quickbird

Output:

wavelength less than 0.25 micron:
let's take s(l)=s(0.25)
...

..

.

wavelength less than 0.25 micron:
let's take s(l)=s(0.25)
Atmospheric correction...
Segmentation fault (core dumped)

I looked into the iwave.cpp, it seems OK. What is the wavelength being less than 0.25 micron, never seen that before...

neteler · November 27, 2016, 7:06pm

On Sun, Nov 27, 2016 at 7:47 PM, Yann <dr.yann.chemin@gmail.com> wrote:

trying to run i.atcorr on an image of Colombo.

i.atcorr --overwrite input=radxs.1 parameters=6s_b1.txt output=refxs.1

...

Segmentation fault (core dumped)

I looked into the iwave.cpp, it seems OK. What is the wavelength being less than 0.25 micron, never seen that before...

I wonder about this:

grep 'ffu.s\[-' imagery/i.atcorr/iwave.cpp
    case 1: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr1[i];
    case 2: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr2[i];
    case 3: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr3[i];
    case 4: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr4[i];
    case 5: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr5[i];

Negative value (-99)? If I remember correctly the Quickbird support
was added still without using the script.

In r69930 (just now) I have added the Quickbird2 filter functions as
CSV which I got from Nikos in August to that you may re-run the
create_iwave.py script and compare the resulting iwave.cpp etc code to
that present in the source code at time.

Markus

ychemin · November 27, 2016, 7:12pm

On 27/11/2016 20:06, Markus Neteler wrote:

On Sun, Nov 27, 2016 at 7:47 PM, Yann <dr.yann.chemin@gmail.com> wrote:

trying to run i.atcorr on an image of Colombo.

i.atcorr --overwrite input=radxs.1 parameters=6s_b1.txt output=refxs.1

...

Segmentation fault (core dumped)

I looked into the iwave.cpp, it seems OK. What is the wavelength being less than 0.25 micron, never seen that before...

I wonder about this:

grep 'ffu.s\[-' imagery/i.atcorr/iwave.cpp
     case 1: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr1[i];
     case 2: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr2[i];
     case 3: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr3[i];
     case 4: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr4[i];
     case 5: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr5[i];

Negative value (-99)? If I remember correctly the Quickbird support
was added still without using the script.

In r69930 (just now) I have added the Quickbird2 filter functions as
CSV which I got from Nikos in August to that you may re-run the
create_iwave.py script and compare the resulting iwave.cpp etc code to
that present in the source code at time.

Markus

Thanks Markus, will look into that.

Also I believe the "less than 0.25" thing comes from either (or both) of these:
static const float wli[5] = {0.001, 0.001, 0.001, 0.001, 0.001};
static const float wls[5] = {0.161, 0.161, 0.161, 0.161, 0.161};

line 2942 in iwave.cpp

ychemin · November 27, 2016, 7:19pm

It seems there is a bug in either the python file or the csv file.

--------

python create_iwave.py sensors_csv/quickbird2.csv

  > Getting sensor name from csv file: quickbird2
  > Number of bands found: 5
Traceback (most recent call last):
   File "create_iwave.py", line 284, in <module>
     main()
   File "create_iwave.py", line 270, in main
     write_cpp(bands, values, sensor, os.path.dirname(inputfile))
   File "create_iwave.py", line 188, in write_cpp
     fi, li = interpolate_band(values[:,[0,b+1]])
   File "create_iwave.py", line 122, in interpolate_band
     filter_f = f(np.arange(start, stop, step))
   File "/usr/lib/python2.7/dist-packages/scipy/interpolate/polyint.py", line 79, in __call__
     y = self._evaluate(x)
   File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py", line 588, in _evaluate
     below_bounds, above_bounds = self._check_bounds(x_new)
   File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py", line 620, in _check_bounds
     raise ValueError("A value in x_new is above the interpolation "
ValueError: A value in x_new is above the interpolation range.

On 27/11/2016 20:06, Markus Neteler wrote:

On Sun, Nov 27, 2016 at 7:47 PM, Yann <dr.yann.chemin@gmail.com> wrote:

trying to run i.atcorr on an image of Colombo.

i.atcorr --overwrite input=radxs.1 parameters=6s_b1.txt output=refxs.1

...

Segmentation fault (core dumped)

I looked into the iwave.cpp, it seems OK. What is the wavelength being less than 0.25 micron, never seen that before...

I wonder about this:

grep 'ffu.s\[-' imagery/i.atcorr/iwave.cpp
     case 1: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr1[i];
     case 2: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr2[i];
     case 3: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr3[i];
     case 4: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr4[i];
     case 5: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr5[i];

Negative value (-99)? If I remember correctly the Quickbird support
was added still without using the script.

In r69930 (just now) I have added the Quickbird2 filter functions as
CSV which I got from Nikos in August to that you may re-run the
create_iwave.py script and compare the resulting iwave.cpp etc code to
that present in the source code at time.

Markus

NikosAlexandris · November 28, 2016, 1:40pm

* Yann <dr.yann.chemin@gmail.com> [2016-11-27 20:12:38 +0100]:

On 27/11/2016 20:06, Markus Neteler wrote:

On Sun, Nov 27, 2016 at 7:47 PM, Yann <dr.yann.chemin@gmail.com> wrote:

trying to run i.atcorr on an image of Colombo.

i.atcorr --overwrite input=radxs.1 parameters=6s_b1.txt output=refxs.1

...

Segmentation fault (core dumped)

I looked into the iwave.cpp, it seems OK. What is the wavelength being less than 0.25 micron, never seen that before...

I wonder about this:

grep 'ffu.s\[-' imagery/i.atcorr/iwave.cpp
     case 1: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr1[i];
     case 2: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr2[i];
     case 3: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr3[i];
     case 4: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr4[i];
     case 5: for(i = 0; i < 65; i++) ffu.s[-99+i] = sr5[i];

Negative value (-99)? If I remember correctly the Quickbird support
was added still without using the script.

In r69930 (just now) I have added the Quickbird2 filter functions as
CSV which I got from Nikos in August to that you may re-run the
create_iwave.py script and compare the resulting iwave.cpp etc code to
that present in the source code at time.

Markus

Thanks Markus, will look into that.

Also I believe the "less than 0.25" thing comes from either (or both) of
these:
  static const float wli[5] = {0.001, 0.001, 0.001, 0.001, 0.001};
  static const float wls[5] = {0.161, 0.161, 0.161, 0.161, 0.161};

line 2942 in iwave.cpp

Back in August, I thoroughly checked running the (modified) python script
against each and every sensor's spectral response data (in form of CSVs)
we have in our repo.

I'll give it a quick check -- I can only work on it after December the
1st though.

Nikos
Back in August, I thoroughly checked running the (modified) script
against each and every sensor's spectral response data (in form of CSVs)
we have in our repo.

I'll give it a quick check -- I can only work on it after December the
1st though.

Nikos

NikosAlexandris · November 28, 2016, 2:55pm

..

Yann:

Also I believe the "less than 0.25" thing comes from either (or both) of
these:
static const float wli[5] = {0.001, 0.001, 0.001, 0.001, 0.001};
static const float wls[5] = {0.161, 0.161, 0.161, 0.161, 0.161};

line 2942 in iwave.cpp

Nikos:

Back in August, I thoroughly checked running the (modified) python script
against each and every sensor's spectral response data (in form of CSVs)
we have in our repo.

I'll give it a quick check -- I can only work on it after December the
1st though.

The uploaded file is wrong in term of units. Please see the attached,
here-in, spectral response values (+ the output template) set in the
correct unit. It works fine for me. I think we had uploaded, back in
August, the "correct" one.

Best, Nikos

(attachments)

quickbird2.csv (7.73 KB)
quickbird2_cpp_template.txt (12.5 KB)

neteler · November 28, 2016, 6:34pm

On Mon, Nov 28, 2016 at 3:55 PM, Nikos Alexandris
<nik@nikosalexandris.net> wrote:
...

Nikos:

Back in August, I thoroughly checked running the (modified) python script
against each and every sensor's spectral response data (in form of CSVs)
we have in our repo.

I'll give it a quick check -- I can only work on it after December the
1st though.

The uploaded file is wrong in term of units. Please see the attached,
here-in, spectral response values (+ the output template) set in the
correct unit. It works fine for me. I think we had uploaded, back in
August, the "correct" one.

No, we didn't upload any Quickbird related CSV in August... see log:
https://trac.osgeo.org/grass/log/grass/trunk/imagery/i.atcorr

But you are right, the first column got messed up in the reformatting
of the CSV file. Fixed in SVN in r69943 (trunk), r69942 (7.2), r69941
(7.0).

The result from python create_iwave.py sensors_csv/quickbird2.csv is
quite different from the "rubbish" definition currently found in
iwave.cpp (originates from r60610).
I have updated the iwave.cpp code accordingly in all 7 branches and trunk.

Yann, others: Please test.

Markus

neteler · December 15, 2016, 10:25pm

Hi Yann,

On Mon, Nov 28, 2016 at 7:34 PM, Markus Neteler <neteler@osgeo.org> wrote:
...

... the first column got messed up in the reformatting
of the CSV file. Fixed in SVN in r69943 (trunk), r69942 (7.2), r69941
(7.0).

...

I have updated the iwave.cpp code accordingly in all 7 branches and trunk.

Yann, others: Please test.

Does i.atcorr now work ok with Quickbird?

Markus

ychemin · December 15, 2016, 11:11pm

Hey Markus,

my attempts have been umsuccessful so far, but i cannot yet say if it is my fault or the module… i have not a good grasp on my dataset yet. will work on it this week end.

cheers,
yann

···

On Dec 15, 2016 11:26 PM, “Markus Neteler” <neteler@osgeo.org> wrote:

Hi Yann,

On Mon, Nov 28, 2016 at 7:34 PM, Markus Neteler <neteler@osgeo.org> wrote:
…

… the first column got messed up in the reformatting
of the CSV file. Fixed in SVN in r69943 (trunk), r69942 (7.2), r69941
(7.0).
…
I have updated the iwave.cpp code accordingly in all 7 branches and trunk.

Yann, others: Please test.

Does i.atcorr now work ok with Quickbird?

Markus