[GRASS-dev] [GRASS GIS] #3979: G79: problem with i.sentinel.import and importer.write_metadata()

GDAL · November 20, 2019, 9:19pm

#3979: G79: problem with i.sentinel.import and importer.write_metadata()
-------------------------------------------------+-------------------------
Reporter: neteler | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone:
Component: Imagery | Version: svn-trunk
Keywords: i.sentinel.import, image | CPU: Unspecified
collections |
Platform: Unspecified |
-------------------------------------------------+-------------------------
I was trying to import Sentinel-2 (S2-L1C) data into G78 in order to make
use of image collections:

{{{
i.sentinel.import input=/mnt/geodata/openeo_bolzano/
register_output=s2_t_register.csv
[...]
Processing <T32TPS_20180608T101021_B02>...
Importing raster map <T32TPS_20180608T101021_B02>...
100%
Processing <T32TPS_20180608T101021_B12>...
Importing raster map <T32TPS_20180608T101021_B12>...
100%
Processing <T32TPS_20180608T101021_B07>...
Importing raster map <T32TPS_20180608T101021_B07>...
100%
Processing <T32TPS_20180608T101021_B11>...
Importing raster map <T32TPS_20180608T101021_B11>...
100%
Processing <T32TPS_20180608T101021_B10>...
Importing raster map <T32TPS_20180608T101021_B10>...
100%
Processing <T32TPS_20180608T101021_B03>...
Importing raster map <T32TPS_20180608T101021_B03>...
100%
Writing metadata to maps...
Traceback (most recent call last):
File "/root/.grass7/addons/scripts/i.sentinel.import", line 504, in
<module>
sys.exit(main())
File "/root/.grass7/addons/scripts/i.sentinel.import", line 488, in main
importer.write_metadata()
File "/root/.grass7/addons/scripts/i.sentinel.import", line 407, in
write_metadata
meta = ip_meta[ip]
KeyError: 'S2B_MSIL1C_20180613T101019_N0206_R022_T32TPS_20180613T122213'
}}}

I am using a rasther fresh version:

{{{
g.version -rge
version=7.9.dev
date=2019
revision=exported
build_date=2019-11-19
build_platform=x86_64-pc-linux-gnu
build_off_t_size=8
libgis_revision=00000
libgis_date="?"
proj=4.9.3
gdal=2.2.3
geos=3.6.2
sqlite=3.22.0
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3979>
GRASS GIS <https://grass.osgeo.org>

Blumentrath_Stefan1 · November 20, 2019, 10:06pm

Sorry, I am on it.

···

Fra: grass-dev grass-dev-bounces@lists.osgeo.org på vegne av GRASS GIS trac@osgeo.org
Sendt: onsdag 20. november 2019 22:19
Kopi: grass-dev@lists.osgeo.org grass-dev@lists.osgeo.org
Emne: [GRASS-dev] [GRASS GIS] #3979: G79: problem with i.sentinel.import and importer.write_metadata()

#3979: G79: problem with i.sentinel.import and importer.write_metadata()
-------------------------------------------------±------------------------
Reporter: neteler | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone:
Component: Imagery | Version: svn-trunk
Keywords: i.sentinel.import, image | CPU: Unspecified
collections |
Platform: Unspecified |
-------------------------------------------------±------------------------
I was trying to import Sentinel-2 (S2-L1C) data into G78 in order to make
use of image collections:

{{{
i.sentinel.import input=/mnt/geodata/openeo_bolzano/
register_output=s2_t_register.csv
[…]
Processing <T32TPS_20180608T101021_B02>…
Importing raster map <T32TPS_20180608T101021_B02>…
100%
Processing <T32TPS_20180608T101021_B12>…
Importing raster map <T32TPS_20180608T101021_B12>…
100%
Processing <T32TPS_20180608T101021_B07>…
Importing raster map <T32TPS_20180608T101021_B07>…
100%
Processing <T32TPS_20180608T101021_B11>…
Importing raster map <T32TPS_20180608T101021_B11>…
100%
Processing <T32TPS_20180608T101021_B10>…
Importing raster map <T32TPS_20180608T101021_B10>…
100%
Processing <T32TPS_20180608T101021_B03>…
Importing raster map <T32TPS_20180608T101021_B03>…
100%
Writing metadata to maps…
Traceback (most recent call last):
File “/root/.grass7/addons/scripts/i.sentinel.import”, line 504, in

sys.exit(main())
File “/root/.grass7/addons/scripts/i.sentinel.import”, line 488, in main
importer.write_metadata()
File “/root/.grass7/addons/scripts/i.sentinel.import”, line 407, in
write_metadata
meta = ip_meta[ip]
KeyError: ‘S2B_MSIL1C_20180613T101019_N0206_R022_T32TPS_20180613T122213’
}}}

I am using a rasther fresh version:

{{{
g.version -rge
version=7.9.dev
date=2019
revision=exported
build_date=2019-11-19
build_platform=x86_64-pc-linux-gnu
build_off_t_size=8
libgis_revision=00000
libgis_date=“?”
proj=4.9.3
gdal=2.2.3
geos=3.6.2
sqlite=3.22.0
}}}

–
Ticket URL: <https://trac.osgeo.org/grass/ticket/3979>
GRASS GIS <https://grass.osgeo.org>

GDAL · November 20, 2019, 11:23pm

#3979: G79: problem with i.sentinel.import and importer.write_metadata()
-------------------------+-------------------------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Imagery | Version: svn-trunk
Resolution: | Keywords: i.sentinel.import, image
       CPU: | collections
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by sbl):

Please test: https://github.com/OSGeo/grass-addons/pull/65
I could not reproduce the error with that particular S2 scene, but got
deprecation warnings from xml on Fedora 30.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3979#comment:1>
GRASS GIS <https://grass.osgeo.org>

GDAL · November 21, 2019, 8:13am

#3979: G79: problem with i.sentinel.import and importer.write_metadata()
-------------------------+-------------------------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Imagery | Version: svn-trunk
Resolution: | Keywords: i.sentinel.import, image
       CPU: | collections
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by neteler):

Thanks for your efforts but still...:

{{{
Importing raster map <T32TPS_20180608T101021_B03>...
  100%
Writing metadata to maps...
Traceback (most recent call last):
   File "/root/.grass7/addons/scripts/i.sentinel.import", line 503, in
<module>
     sys.exit(main())
   File "/root/.grass7/addons/scripts/i.sentinel.import", line 487, in main
     importer.write_metadata()
   File "/root/.grass7/addons/scripts/i.sentinel.import", line 406, in
write_metadata
     meta = ip_meta[ip]
KeyError: 'S2B_MSIL1C_20180613T101019_N0206_R022_T32TPS_20180613T122213'
GRASS 7.9.dev (utm32n_32632): >
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3979#comment:2>
GRASS GIS <https://grass.osgeo.org>

GDAL · November 21, 2019, 9:44am

#3979: G79: problem with i.sentinel.import and importer.write_metadata()
-------------------------+-------------------------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Imagery | Version: svn-trunk
Resolution: | Keywords: i.sentinel.import, image
       CPU: | collections
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by neteler):

I added the pdb debugger there:

{{{
> /root/.grass7/addons/scripts/i.sentinel.import(407)write_metadata()
-> meta = ip_meta[ip]
(Pdb) ip
'S2A_MSIL1C_20180608T101021_N0206_R022_T32TPS_20180608T135059'
(Pdb) ip_meta[ip]
{'timestamp': datetime.datetime(2018, 6, 8, 10, 17, 26, 568000),
'SATELLITE': 'S2A', 'CLOUDY_PIXEL_PERCENTAGE': '63.2363',
'DEGRADED_MSI_DATA_PERCENTAGE': '0', 'ZENITH_ANGLE_0': '6.87463083682177',
'AZIMUTH_ANGLE_0': '106.678480338394', 'ZENITH_ANGLE_9':
'6.92095256322843', 'AZIMUTH_ANGLE_9': '106.766469213864',
'ZENITH_ANGLE_10': '6.6711378900598', 'AZIMUTH_ANGLE_10':
'106.075220597716', 'ZENITH_ANGLE_1': '6.60472703969214',
'AZIMUTH_ANGLE_1': '105.531721669131', 'ZENITH_ANGLE_2':
'6.64725248700114', 'AZIMUTH_ANGLE_2': '105.80797975942',
'ZENITH_ANGLE_3': '6.69114850385045', 'AZIMUTH_ANGLE_3':
'106.033552825819', 'ZENITH_ANGLE_4': '6.72593186902208',
'AZIMUTH_ANGLE_4': '106.198282849613', 'ZENITH_ANGLE_5':
'6.76337836610513', 'AZIMUTH_ANGLE_5': '106.292239966037',
'ZENITH_ANGLE_6': '6.79969827918603', 'AZIMUTH_ANGLE_6':
'106.408451382071', 'ZENITH_ANGLE_7': '6.62125461003938',
'AZIMUTH_ANGLE_7': '105.678569152871', 'ZENITH_ANGLE_8':
'6.83898070776319', 'AZIMUTH_ANGLE_8': '106.505396826378',
'ZENITH_ANGLE_11': '6.74365826507417', 'AZIMUTH_ANGLE_11':
'106.342330082536', 'ZENITH_ANGLE_12': '6.83298026030892',
'AZIMUTH_ANGLE_12': '106.566283022332', 'MEAN_SUN_ZENITH_GRID_ANGLE':
26.24232325141777, 'MEAN_SUN_AZIMUTH_GRID_ANGLE': 149.15294328922496,
'MEAN_SUN_ZENITH_ANGLE': '26.2423240875397', 'MEAN_SUN_AZIMUTH_ANGLE':
'149.1529063561'}
}}}

No idea - I don't see the issue

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3979#comment:3>
GRASS GIS <https://grass.osgeo.org>

GDAL · November 21, 2019, 10:22am

#3979: G79: problem with i.sentinel.import and importer.write_metadata()
-------------------------+-------------------------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Imagery | Version: svn-trunk
Resolution: | Keywords: i.sentinel.import, image
       CPU: | collections
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by sbl):

That`s odd indeed. I cannot reproduce the error with exactly the same
scene and both trunk and 7.8 on Fedora 30.

With an older version of trunk:
{{{
g.version -rge
version=7.9.dev
date=2019
revision=400349708
build_date=2019-08-31
build_platform=x86_64-pc-linux-gnu
build_off_t_size=8
libgis_revision=00000
libgis_date="?"
proj4=5.2.0
gdal=2.3.2
geos=3.7.1
sqlite=3.26.0
}}}

as well as a fresh compiled one and 7.8.1:
{{{
g.version -rge
version=7.8.1RC1
date=2019
revision=exported
build_date=2019-11-11
build_platform=x86_64-redhat-linux-gnu
build_off_t_size=8
libgis_revision=00000
libgis_date="?"
proj=5.2.0
gdal=2.3.2
geos=3.7.1
sqlite=3.26.0
}}}

I have no issues with i.sentinel.import with that scene.

Maybe the download was not 100% OK?
Do you have other scenes in that directory?

BTW, does 7.8 support band references?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3979#comment:4>
GRASS GIS <https://grass.osgeo.org>

GDAL · November 21, 2019, 1:33pm

#3979: G79: problem with i.sentinel.import and importer.write_metadata()
-------------------------+-------------------------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Imagery | Version: svn-trunk
Resolution: | Keywords: i.sentinel.import, image
       CPU: | collections
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by neteler):

My system here is based on the mundialis GRASS GIS 7.9/PDAL/Python 3
docker image (Ubuntu).

I have added, since it is a key error, added a print statement:

{{{
WARNING: Raster map <T32TPS_20180608T101021_B03> already exists and will
be
          overwritten
Importing raster map <T32TPS_20180608T101021_B03>...
  100%
Writing metadata to maps...
dict_keys(['timestamp', 'SATELLITE', 'CLOUDY_PIXEL_PERCENTAGE',
'DEGRADED_MSI_DATA_PERCENTAGE', 'ZENITH_ANGLE_0', 'AZIMUTH_ANGLE_0',
'ZENITH_ANGLE_9', 'AZIMUTH_ANGLE_9', 'ZENITH_ANGLE_10', '
AZIMUTH_ANGLE_10', 'ZENITH_ANGLE_1', 'AZIMUTH_ANGLE_1', 'ZENITH_ANGLE_2',
'AZIMUTH_ANGLE_2', 'ZENITH_ANGLE_3', 'AZIMUTH_ANGLE_3', 'ZENITH_ANGLE_4',
'AZIMUTH_ANGLE_4', 'ZENITH_ANGLE_5', 'AZIM
UTH_ANGLE_5', 'ZENITH_ANGLE_6', 'AZIMUTH_ANGLE_6', 'ZENITH_ANGLE_7',
'AZIMUTH_ANGLE_7', 'ZENITH_ANGLE_8', 'AZIMUTH_ANGLE_8', 'ZENITH_ANGLE_11',
'AZIMUTH_ANGLE_11', 'ZENITH_ANGLE_12', 'AZIMUT
H_ANGLE_12', 'MEAN_SUN_ZENITH_GRID_ANGLE', 'MEAN_SUN_AZIMUTH_GRID_ANGLE',
'MEAN_SUN_ZENITH_ANGLE', 'MEAN_SUN_AZIMUTH_ANGLE'])
dict_keys(['timestamp', 'SATELLITE', 'CLOUDY_PIXEL_PERCENTAGE',
'DEGRADED_MSI_DATA_PERCENTAGE', 'ZENITH_ANGLE_0', 'AZIMUTH_ANGLE_0',
'ZENITH_ANGLE_9', 'AZIMUTH_ANGLE_9', 'ZENITH_ANGLE_10', '
AZIMUTH_ANGLE_10', 'ZENITH_ANGLE_1', 'AZIMUTH_ANGLE_1', 'ZENITH_ANGLE_2',
'AZIMUTH_ANGLE_2', 'ZENITH_ANGLE_3', 'AZIMUTH_ANGLE_3', 'ZENITH_ANGLE_4',
'AZIMUTH_ANGLE_4', 'ZENITH_ANGLE_5', 'AZIM
UTH_ANGLE_5', 'ZENITH_ANGLE_6', 'AZIMUTH_ANGLE_6', 'ZENITH_ANGLE_7',
'AZIMUTH_ANGLE_7', 'ZENITH_ANGLE_8', 'AZIMUTH_ANGLE_8', 'ZENITH_ANGLE_11',
'AZIMUTH_ANGLE_11', 'ZENITH_ANGLE_12', 'AZIMUT
H_ANGLE_12', 'MEAN_SUN_ZENITH_GRID_ANGLE', 'MEAN_SUN_AZIMUTH_GRID_ANGLE',
'MEAN_SUN_ZENITH_ANGLE', 'MEAN_SUN_AZIMUTH_ANGLE'])
dict_keys(['timestamp', 'SATELLITE', 'CLOUDY_PIXEL_PERCENTAGE',
'DEGRADED_MSI_DATA_PERCENTAGE', 'ZENITH_ANGLE_0', 'AZIMUTH_ANGLE_0',
'ZENITH_ANGLE_9', 'AZIMUTH_ANGLE_9', 'ZENITH_ANGLE_10', '
AZIMUTH_ANGLE_10', 'ZENITH_ANGLE_1', 'AZIMUTH_ANGLE_1', 'ZENITH_ANGLE_2',
'AZIMUTH_ANGLE_2', 'ZENITH_ANGLE_3', 'AZIMUTH_ANGLE_3', 'ZENITH_ANGLE_4',
'AZIMUTH_ANGLE_4', 'ZENITH_ANGLE_5', 'AZIM
UTH_ANGLE_5', 'ZENITH_ANGLE_6', 'AZIMUTH_ANGLE_6', 'ZENITH_ANGLE_7',
'AZIMUTH_ANGLE_7', 'ZENITH_ANGLE_8', 'AZIMUTH_ANGLE_8', 'ZENITH_ANGLE_11',
'AZIMUTH_ANGLE_11', 'ZENITH_ANGLE_12', 'AZIMUT
H_ANGLE_12', 'MEAN_SUN_ZENITH_GRID_ANGLE', 'MEAN_SUN_AZIMUTH_GRID_ANGLE',
'MEAN_SUN_ZENITH_ANGLE', 'MEAN_SUN_AZIMUTH_ANGLE'])
dict_keys(['timestamp', 'SATELLITE', 'CLOUDY_PIXEL_PERCENTAGE',
'DEGRADED_MSI_DATA_PERCENTAGE', 'ZENITH_ANGLE_0', 'AZIMUTH_ANGLE_0',
'ZENITH_ANGLE_9', 'AZIMUTH_ANGLE_9', 'ZENITH_ANGLE_10', '
AZIMUTH_ANGLE_10', 'ZENITH_ANGLE_1', 'AZIMUTH_ANGLE_1', 'ZENITH_ANGLE_2',
'AZIMUTH_ANGLE_2', 'ZENITH_ANGLE_3', 'AZIMUTH_ANGLE_3', 'ZENITH_ANGLE_4',
'AZIMUTH_ANGLE_4', 'ZENITH_ANGLE_5', 'AZIM
UTH_ANGLE_5', 'ZENITH_ANGLE_6', 'AZIMUTH_ANGLE_6', 'ZENITH_ANGLE_7',
'AZIMUTH_ANGLE_7', 'ZENITH_ANGLE_8', 'AZIMUTH_ANGLE_8', 'ZENITH_ANGLE_11',
'AZIMUTH_ANGLE_11', 'ZENITH_ANGLE_12', 'AZIMUT
H_ANGLE_12', 'MEAN_SUN_ZENITH_GRID_ANGLE', 'MEAN_SUN_AZIMUTH_GRID_ANGLE',
'MEAN_SUN_ZENITH_ANGLE', 'MEAN_SUN_AZIMUTH_ANGLE'])
[...]
dict_keys(['timestamp', 'SATELLITE', 'CLOUDY_PIXEL_PERCENTAGE',
'DEGRADED_MSI_DATA_PERCENTAGE', 'ZENITH_ANGLE_0', 'AZIMUTH_ANGLE_0',
'ZENITH_ANGLE_9', 'AZIMUTH_ANGLE_9', 'ZENITH_ANGLE_10',
'AZIMUTH_ANGLE_10', 'ZENITH_ANGLE_1', 'AZIMUTH_ANGLE_1', 'ZENITH_ANGLE_2',
'AZIMUTH_ANGLE_2', 'ZENITH_ANGLE_3', 'AZIMUTH_ANGLE_3', 'ZENITH_ANGLE_4',
'AZIMUTH_ANGLE_4', 'ZENITH_ANGLE_5', 'AZIMUTH_ANGLE_5', 'ZENITH_ANGLE_6',
'AZIMUTH_ANGLE_6', 'ZENITH_ANGLE_7', 'AZIMUTH_ANGLE_7', 'ZENITH_ANGLE_8',
'AZIMUTH_ANGLE_8', 'ZENITH_ANGLE_11', 'AZIMUTH_ANGLE_11',
'ZENITH_ANGLE_12', 'AZIMUTH_ANGLE_12', 'MEAN_SUN_ZENITH_GRID_ANGLE',
'MEAN_SUN_AZIMUTH_GRID_ANGLE', 'MEAN_SUN_ZENITH_ANGLE',
'MEAN_SUN_AZIMUTH_ANGLE'])
Traceback (most recent call last):
   File "/root/.grass7/addons/scripts/i.sentinel.import", line 505, in
<module>
     sys.exit(main())
   File "/root/.grass7/addons/scripts/i.sentinel.import", line 489, in main
     importer.write_metadata()
   File "/root/.grass7/addons/scripts/i.sentinel.import", line 407, in
write_metadata
     meta = ip_meta[ip]
KeyError: 'S2B_MSIL1C_20180613T101019_N0206_R022_T32TPS_20180613T122213'
}}}

PS: No, 7.8.x does not contain image collection support.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3979#comment:5>
GRASS GIS <https://grass.osgeo.org>

GDAL · November 21, 2019, 2:30pm

#3979: G79: problem with i.sentinel.import and importer.write_metadata()
-------------------------+-------------------------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Imagery | Version: svn-trunk
Resolution: | Keywords: i.sentinel.import, image
       CPU: | collections
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by martinl):

What is content of `/mnt/geodata/openeo_bolzano/` dir?

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3979#comment:6>
GRASS GIS <https://grass.osgeo.org>

GDAL · November 21, 2019, 3:54pm

#3979: G79: problem with i.sentinel.import and importer.write_metadata()
-------------------------+-------------------------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Imagery | Version: svn-trunk
Resolution: | Keywords: i.sentinel.import, image
       CPU: | collections
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by neteler):

Replying to [comment:6 martinl]:
> What is content of `/mnt/geodata/openeo_bolzano/` dir?

{{{
root@d07d7993f8ce:/src/actinia_core# ls -la /mnt/geodata/openeo_bolzano/
total 5083276
drwxr-xr-x 7 root root 4096 Nov 21 10:42 .
drwxr-xr-x 13 1002 1002 4096 Nov 19 12:09 ..
-rw-r--r-- 1 1002 1002 98304 Nov 19 12:07 openeo_bolzano.gpkg
-rw-r--r-- 1 root root 778891069 Nov 19 12:10
S2A_MSIL1C_20180608T101021_N0206_R022_T32TPS_20180608T135059.zip
-rw-r--r-- 1 root root 666971053 Nov 19 12:10
S2A_MSIL1C_20180611T102021_N0206_R065_T32TPS_20180611T123241.zip
drwxr-xr-x 7 root root 4096 Nov 19 12:31
S2A_MSIL1C_20180618T101021_N0206_R022_T32TPS_20180618T135619.SAFE
-rw-r--r-- 1 root root 855084777 Nov 19 12:10
S2A_MSIL1C_20180618T101021_N0206_R022_T32TPS_20180618T135619.zip
drwxr-xr-x 7 root root 4096 Nov 19 12:31
S2A_MSIL1C_20180621T102021_N0206_R065_T32TPS_20180621T140615.SAFE
-rw-r--r-- 1 root root 800185372 Nov 19 12:10
S2A_MSIL1C_20180621T102021_N0206_R065_T32TPS_20180621T140615.zip
drwxr-xr-x 7 root root 4096 Nov 19 12:31
S2B_MSIL1C_20180606T102019_N0206_R065_T32TPS_20180606T172808.SAFE
-rw-r--r-- 1 root root 730499111 Nov 19 12:10
S2B_MSIL1C_20180606T102019_N0206_R065_T32TPS_20180606T172808.zip
drwxr-xr-x 5 root root 4096 Nov 19 12:34
S2B_MSIL1C_20180613T101019_N0206_R022_T32TPS_20180613T122213.SAFE
-rw-r--r-- 1 root root 574563063 Nov 19 12:11
S2B_MSIL1C_20180613T101019_N0206_R022_T32TPS_20180613T122213.zip
drwxr-xr-x 7 root root 4096 Nov 19 12:31
S2B_MSIL1C_20180616T102019_N0206_R065_T32TPS_20180616T154713.SAFE
-rw-r--r-- 1 root root 798910322 Nov 19 12:11
S2B_MSIL1C_20180616T102019_N0206_R065_T32TPS_20180616T154713.zip
-rw-r--r-- 1 root root 434 Nov 19 11:58 s2_L1C_scenes.csv
}}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3979#comment:7>
GRASS GIS <https://grass.osgeo.org>

GDAL · November 21, 2019, 7:45pm

#3979: G79: problem with i.sentinel.import and importer.write_metadata()
-------------------------+-------------------------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Imagery | Version: svn-trunk
Resolution: | Keywords: i.sentinel.import, image
       CPU: | collections
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by sbl):

Hm, I think I have a hunch. Can you try to
1) delete all SAFE directories (or use a different unzip dir)
2) run i.sentinel.import again and see if the error remains?

If you don`t link (l-flag) the unpacked SAFE dirs should actually be
removed automatically...

If they are present, i.sentinel.import does not unpack again (anymore
after my changes). If an import is interrupted it may lead to issues like
yours...

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3979#comment:8>
GRASS GIS <https://grass.osgeo.org>

GDAL · November 21, 2019, 9:12pm

#3979: G79: problem with i.sentinel.import and importer.write_metadata()
-------------------------+-------------------------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Imagery | Version: svn-trunk
Resolution: | Keywords: i.sentinel.import, image
       CPU: | collections
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by neteler):

Replying to [comment:8 sbl]:
> Hm, I think I have a hunch. Can you try to
> 1) delete all SAFE directories (or use a different unzip dir)
> 2) run i.sentinel.import again and see if the error remains?

Bingo! Now it ran through, finally.

> If you don`t link (l-flag) the unpacked SAFE dirs should actually be
removed automatically...

I didn't use -l this time.

> If they are present, i.sentinel.import does not unpack again (anymore
after my changes). If an import is interrupted it may lead to issues like
yours...

I see. May I suggest that, if present, the SAFE directory be actually
removed before unpacking? Less confusion

Rationale: it may happen then a `i.sentinel.import` job is interrupted, be
by the user, for disk-full reasons or whatever else might occur...

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3979#comment:9>
GRASS GIS <https://grass.osgeo.org>

GDAL · November 22, 2019, 8:15pm

#3979: G79: problem with i.sentinel.import and importer.write_metadata()
-------------------------+-------------------------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Imagery | Version: svn-trunk
Resolution: | Keywords: i.sentinel.import, image
       CPU: | collections
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by sbl):

OK, I see three options:
1) revert to old behavior
2) add a flag for not unziping again
3) use GDALs /vsizip/ virtual file system (no unzipping)

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3979#comment:10>
GRASS GIS <https://grass.osgeo.org>

GDAL · November 23, 2019, 10:49pm

#3979: G79: problem with i.sentinel.import and importer.write_metadata()
-------------------------+-------------------------------------------------
  Reporter: neteler | Owner: grass-dev@…
      Type: defect | Status: new
  Priority: normal | Milestone:
Component: Imagery | Version: svn-trunk
Resolution: | Keywords: i.sentinel.import, image
       CPU: | collections
  Unspecified | Platform: Unspecified
-------------------------+-------------------------------------------------

Comment (by sbl):

OK, for the time being I solved it with a new flag (-n) for not unzipping
again. So old behavior is restored (see updated PR:
https://github.com/OSGeo/grass-addons/pull/65).

However, I would prefer using /vsizip// and reading from zip as a default
instead of unpacking at all. That would save disk space and be more
efficient in many (most?) cases.
Unless one wants to link data and use them frequently (e.g. linking L2A
products and use them without any further pre-processing) repeatedly...

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3979#comment:11>
GRASS GIS <https://grass.osgeo.org>