[GRASS-user] i.landsat.toar unable to read metadata file

Hi,

I am using GRASS 7.0.svn (SVN Revision: 53972) in Windows 7 64-bit, trying to generate cloud masks for Landsat5 and 7 images acquired from USGS.
For some reason i.landsat.toar gives me "Failed to identify satellite" error when I try to run it with the metadata file loaded. I'm able to load the contents of the metadata file to the viewer of the application, but apparently the tool can't find the required information.

In the case above I've loaded the file * _MTLold.txt. If *_MTL.txt file is loaded, I got the following messages:
WARNING: Unable to recognize satellite platform [0]
ERROR: Failed to identify satellite
Product date is 2012-10-09.

Any help would be much appreciated!

Andras

On 04/12/12 08:05, Balázs András (METLA) wrote:

Hi,

I am using GRASS 7.0.svn (SVN Revision: 53972) in Windows 7 64-bit, trying to generate cloud masks for Landsat5 and 7 images acquired from USGS.
For some reason i.landsat.toar gives me "Failed to identify satellite" error when I try to run it with the metadata file loaded. I'm able to load the contents of the metadata file to the viewer of the application, but apparently the tool can't find the required information.

In the case above I've loaded the file * _MTLold.txt. If *_MTL.txt file is loaded, I got the following messages:
WARNING: Unable to recognize satellite platform [0]
ERROR: Failed to identify satellite
Product date is 2012-10-09.

I've noticed that the MTL files have some form of binary info at the end (last line). This keeps several text editors from opening these files. When I open them in vi and erase the last line, everything is fine. I don't know where that line comes from.

Maybe that's what's keeping the module from reading them correctly ?

Moritz

Hi Moritz,

thanks for your input!
I thought of this myself. Removed the binary bit from the files and used them text-only versions without success.
It has to do something with the structure of the metadata files, as Jorge Tizado pointed it out for me.

Greets,

Andras

-----Original Message-----
From: Moritz Lennert [mailto:mlennert@club.worldonline.be]
Sent: 11. joulukuuta 2012 11:57
To: Balázs András (METLA)
Cc: grass-user@lists.osgeo.org
Subject: Re: [GRASS-user] i.landsat.toar unable to read metadata file

On 04/12/12 08:05, Balázs András (METLA) wrote:

Hi,

I am using GRASS 7.0.svn (SVN Revision: 53972) in Windows 7 64-bit, trying to generate cloud masks for Landsat5 and 7 images acquired from USGS.
For some reason i.landsat.toar gives me "Failed to identify satellite" error when I try to run it with the metadata file loaded. I'm able to load the contents of the metadata file to the viewer of the application, but apparently the tool can't find the required information.

In the case above I've loaded the file * _MTLold.txt. If *_MTL.txt file is loaded, I got the following messages:
WARNING: Unable to recognize satellite platform [0]
ERROR: Failed to identify satellite
Product date is 2012-10-09.

I've noticed that the MTL files have some form of binary info at the end (last line). This keeps several text editors from opening these files.
When I open them in vi and erase the last line, everything is fine. I don't know where that line comes from.

Maybe that's what's keeping the module from reading them correctly ?

Moritz

Balázs András:

..

I thought of this myself. Removed the binary bit from the files and used
them text-only versions without success. It has to do something with the
structure of the metadata files, as Jorge Tizado pointed it out for me.

Same problem here, using GRASS 6.4.3svn (2012). And it doesn't work by using
the *MTLold.txt file(s) as well.

Thanks, Nikos

Here the solution!

cheers
m

---------- Forwarded message ----------
From: E. Jorge Tizado <ej.tizado@unileon.es>
Date: Mon, Dec 10, 2012 at 11:43 AM
Subject: Re: Fwd: [GRASS-user] i.landsat.toar unable to read metadata file
To: Markus Neteler <neteler@osgeo.org>

Hi Markus,

The Andras problem is solved!
The problem arises in another new! change of the 'standard' metadata
file MTL.txt; now use LANDSAT_7 instead of Landsat7 as spacecraft_id

I modify the code to avoid this problem with other users;
I attach the file landsat_met.c for replacement in SVN servers (grass6
and grass_trunk)

Best

    Jorge

Hi Jorge,

do you have a possibility to help this user?

thanks
Markus

--
+---------------------------------------------------------+
| Dr. E. Jorge Tizado
| Dpto. Biodiversidad y Gestión Ambiental: Zoología
| Universidad de León - Campus de Ponferrada
| E-24400 Ponferrada (León, España)
+---------------------------------------------------------+
| tfno.: 987 442 018 - 696 138 405
| e-mail: ej.tizado@unileon.es
| web: http://www3.unileon.es/personal/wwdbajtm/index.php
+---------------------------------------------------------+

(attachments)

landsat_met.c (7.2 KB)

(Apologies for not replying to the latest e-mail in this thread -- posting
from another machine with no access to past threads)

I tested the modified landsat_met.c code, provided by E. Jorge Tizado. It
works! Yet, it fails somewhere else, e.g.:

--%<---
# get toar
i.landsat.toar input_prefix=${BASE} output_prefix=${BASE}_toar
metfile=${BASE}_MTL.txt

ERROR: Unknown production date (defined by 'product_date')
--->%--

It seems there are several modifications to the MET content structure.
Fortunately (?) two MET files are provided *currently* (at least I have seen
this in several L[MTE][457] scenes.

For example, an LT5 packed scene contains:
..
LT51810332008141MTI00_MTL.txt
LT51810332008141MTI00_MTLold.txt
..

The diff file looks like:

--%<---
--- LT51810332008141MTI00_MTL.txt 2012-12-12 18:47:35.393026898 +0200
+++ LT51810332008141MTI00_MTLold.txt 2012-12-05 08:42:36.000000000 +0200
@@ -2,144 +2,140 @@
   GROUP = METADATA_FILE_INFO
     ORIGIN = "Image courtesy of the U.S. Geological Survey"
     REQUEST_ID = "0101212018233_00067"
- LANDSAT_SCENE_ID = "LT51810332008141MTI00"
- FILE_DATE = 2012-12-05T06:34:04Z
+ PRODUCT_CREATION_TIME = 2012-12-05T06:42:36Z
     STATION_ID = "EDC"
- PROCESSING_SOFTWARE_VERSION = "LPGS_12.1.3"
- DATA_CATEGORY = "NOMINAL"
+ LANDSAT5_XBAND = "1"
+ GROUND_STATION = "MTI"
+ LPS_PROCESSOR_NUMBER = 0
+ DATEHOUR_CONTACT_PERIOD = "0814108"
+ SUBINTERVAL_NUMBER = "01"
   END_GROUP = METADATA_FILE_INFO
   GROUP = PRODUCT_METADATA
- DATA_TYPE = "L1T"
- DATA_TYPE_L0RP = "TMR_L0RP"
+ PRODUCT_TYPE = "L1T"
     ELEVATION_SOURCE = "GLS2000"
- OUTPUT_FORMAT = "GEOTIFF"
+ PROCESSING_SOFTWARE = "LPGS_12.1.3"
     EPHEMERIS_TYPE = "DEFINITIVE"
- SPACECRAFT_ID = "LANDSAT_5"
+ SPACECRAFT_ID = "Landsat5"
     SENSOR_ID = "TM"
     SENSOR_MODE = "BUMPER"
+ ACQUISITION_DATE = 2008-05-20
+ SCENE_CENTER_SCAN_TIME = 08:40:46.1460440Z
     WRS_PATH = 181
- WRS_ROW = 033
- DATE_ACQUIRED = 2008-05-20
- SCENE_CENTER_TIME = 08:40:46.1460440Z
- CORNER_UL_LAT_PRODUCT = 39.86353
- CORNER_UL_LON_PRODUCT = 25.17729
- CORNER_UR_LAT_PRODUCT = 39.87362
- CORNER_UR_LON_PRODUCT = 27.99044
- CORNER_LL_LAT_PRODUCT = 37.92070
- CORNER_LL_LON_PRODUCT = 25.22630
- CORNER_LR_LAT_PRODUCT = 37.93012
- CORNER_LR_LON_PRODUCT = 27.96380
- CORNER_UL_PROJECTION_X_PRODUCT = 344100.000
- CORNER_UL_PROJECTION_Y_PRODUCT = 4414200.000
- CORNER_UR_PROJECTION_X_PRODUCT = 584700.000
- CORNER_UR_PROJECTION_Y_PRODUCT = 4414200.000
- CORNER_LL_PROJECTION_X_PRODUCT = 344100.000
- CORNER_LL_PROJECTION_Y_PRODUCT = 4198500.000
- CORNER_LR_PROJECTION_X_PRODUCT = 584700.000
- CORNER_LR_PROJECTION_Y_PRODUCT = 4198500.000
- REFLECTIVE_LINES = 7191
- REFLECTIVE_SAMPLES = 8021
- THERMAL_LINES = 7191
- THERMAL_SAMPLES = 8021
- FILE_NAME_BAND_1 = "LT51810332008141MTI00_B1.TIF"
- FILE_NAME_BAND_2 = "LT51810332008141MTI00_B2.TIF"
- FILE_NAME_BAND_3 = "LT51810332008141MTI00_B3.TIF"
- FILE_NAME_BAND_4 = "LT51810332008141MTI00_B4.TIF"
- FILE_NAME_BAND_5 = "LT51810332008141MTI00_B5.TIF"
- FILE_NAME_BAND_6 = "LT51810332008141MTI00_B6.TIF"
- FILE_NAME_BAND_7 = "LT51810332008141MTI00_B7.TIF"
- GROUND_CONTROL_POINT_FILE_NAME = "LT51810332008141MTI00_GCP.txt"
- REPORT_VERIFY_FILE_NAME = "LT51810332008141MTI00_VER.txt"
- BROWSE_VERIFY_FILE_NAME = "LT51810332008141MTI00_VER.jpg"
- METADATA_FILE_NAME = "LT51810332008141MTI00_MTL.txt"
- CPF_NAME = "L5CPF20080401_20080630.16"
+ STARTING_ROW = 33
+ ENDING_ROW = 33
+ BAND_COMBINATION = "1234567"
+ PRODUCT_UL_CORNER_LAT = 39.8635274
+ PRODUCT_UL_CORNER_LON = 25.1772902
+ PRODUCT_UR_CORNER_LAT = 39.8736219
+ PRODUCT_UR_CORNER_LON = 27.9904391
+ PRODUCT_LL_CORNER_LAT = 37.9207012
+ PRODUCT_LL_CORNER_LON = 25.2262979
+ PRODUCT_LR_CORNER_LAT = 37.9301230
+ PRODUCT_LR_CORNER_LON = 27.9637968
+ PRODUCT_UL_CORNER_MAPX = 344100.000
+ PRODUCT_UL_CORNER_MAPY = 4414200.000
+ PRODUCT_UR_CORNER_MAPX = 584700.000
+ PRODUCT_UR_CORNER_MAPY = 4414200.000
+ PRODUCT_LL_CORNER_MAPX = 344100.000
+ PRODUCT_LL_CORNER_MAPY = 4198500.000
+ PRODUCT_LR_CORNER_MAPX = 584700.000
+ PRODUCT_LR_CORNER_MAPY = 4198500.000
+ PRODUCT_SAMPLES_REF = 8021
+ PRODUCT_LINES_REF = 7191
+ PRODUCT_SAMPLES_THM = 8021
+ PRODUCT_LINES_THM = 7191
+ BAND1_FILE_NAME = "LT51810332008141MTI00_B1.TIF"
+ BAND2_FILE_NAME = "LT51810332008141MTI00_B2.TIF"
+ BAND3_FILE_NAME = "LT51810332008141MTI00_B3.TIF"
+ BAND4_FILE_NAME = "LT51810332008141MTI00_B4.TIF"
+ BAND5_FILE_NAME = "LT51810332008141MTI00_B5.TIF"
+ BAND6_FILE_NAME = "LT51810332008141MTI00_B6.TIF"
+ BAND7_FILE_NAME = "LT51810332008141MTI00_B7.TIF"
+ GCP_FILE_NAME = "LT51810332008141MTI00_GCP.txt"
+ VER_REPORT_FILE_NAME = "LT51810332008141MTI00_VER.txt"
+ VER_BROWSE_FILE_NAME = "LT51810332008141MTI00_VER.jpg"
+ METADATA_L1_FILE_NAME = "LT51810332008141MTI00_MTLold.txt"
+ CPF_FILE_NAME = "L5CPF20080401_20080630_16"
   END_GROUP = PRODUCT_METADATA
- GROUP = IMAGE_ATTRIBUTES
- CLOUD_COVER = 10.00
- IMAGE_QUALITY = 9
- SUN_AZIMUTH = 127.92027132
- SUN_ELEVATION = 63.07405928
- GROUND_CONTROL_POINTS_MODEL = 1
- GEOMETRIC_RMSE_MODEL = 3.741
- GEOMETRIC_RMSE_MODEL_Y = 2.933
- GEOMETRIC_RMSE_MODEL_X = 2.323
- GROUND_CONTROL_POINTS_VERIFY = 1265
- GEOMETRIC_RMSE_VERIFY = 0.160
- END_GROUP = IMAGE_ATTRIBUTES
   GROUP = MIN_MAX_RADIANCE
- RADIANCE_MAXIMUM_BAND_1 = 193.000
- RADIANCE_MINIMUM_BAND_1 = -1.520
- RADIANCE_MAXIMUM_BAND_2 = 365.000
- RADIANCE_MINIMUM_BAND_2 = -2.840
- RADIANCE_MAXIMUM_BAND_3 = 264.000
- RADIANCE_MINIMUM_BAND_3 = -1.170
- RADIANCE_MAXIMUM_BAND_4 = 221.000
- RADIANCE_MINIMUM_BAND_4 = -1.510
- RADIANCE_MAXIMUM_BAND_5 = 30.200
- RADIANCE_MINIMUM_BAND_5 = -0.370
- RADIANCE_MAXIMUM_BAND_6 = 15.303
- RADIANCE_MINIMUM_BAND_6 = 1.238
- RADIANCE_MAXIMUM_BAND_7 = 16.500
- RADIANCE_MINIMUM_BAND_7 = -0.150
+ LMAX_BAND1 = 193.000
+ LMIN_BAND1 = -1.520
+ LMAX_BAND2 = 365.000
+ LMIN_BAND2 = -2.840
+ LMAX_BAND3 = 264.000
+ LMIN_BAND3 = -1.170
+ LMAX_BAND4 = 221.000
+ LMIN_BAND4 = -1.510
+ LMAX_BAND5 = 30.200
+ LMIN_BAND5 = -0.370
+ LMAX_BAND6 = 15.303
+ LMIN_BAND6 = 1.238
+ LMAX_BAND7 = 16.500
+ LMIN_BAND7 = -0.150
   END_GROUP = MIN_MAX_RADIANCE
   GROUP = MIN_MAX_PIXEL_VALUE
- QUANTIZE_CAL_MAX_BAND_1 = 255
- QUANTIZE_CAL_MIN_BAND_1 = 1
- QUANTIZE_CAL_MAX_BAND_2 = 255
- QUANTIZE_CAL_MIN_BAND_2 = 1
- QUANTIZE_CAL_MAX_BAND_3 = 255
- QUANTIZE_CAL_MIN_BAND_3 = 1
- QUANTIZE_CAL_MAX_BAND_4 = 255
- QUANTIZE_CAL_MIN_BAND_4 = 1
- QUANTIZE_CAL_MAX_BAND_5 = 255
- QUANTIZE_CAL_MIN_BAND_5 = 1
- QUANTIZE_CAL_MAX_BAND_6 = 255
- QUANTIZE_CAL_MIN_BAND_6 = 1
- QUANTIZE_CAL_MAX_BAND_7 = 255
- QUANTIZE_CAL_MIN_BAND_7 = 1
+ QCALMAX_BAND1 = 255.0
+ QCALMIN_BAND1 = 1.0
+ QCALMAX_BAND2 = 255.0
+ QCALMIN_BAND2 = 1.0
+ QCALMAX_BAND3 = 255.0
+ QCALMIN_BAND3 = 1.0
+ QCALMAX_BAND4 = 255.0
+ QCALMIN_BAND4 = 1.0
+ QCALMAX_BAND5 = 255.0
+ QCALMIN_BAND5 = 1.0
+ QCALMAX_BAND6 = 255.0
+ QCALMIN_BAND6 = 1.0
+ QCALMAX_BAND7 = 255.0
+ QCALMIN_BAND7 = 1.0
   END_GROUP = MIN_MAX_PIXEL_VALUE
   GROUP = PRODUCT_PARAMETERS
- CORRECTION_GAIN_BAND_1 = "CPF"
- CORRECTION_GAIN_BAND_2 = "CPF"
- CORRECTION_GAIN_BAND_3 = "CPF"
- CORRECTION_GAIN_BAND_4 = "CPF"
- CORRECTION_GAIN_BAND_5 = "CPF"
- CORRECTION_GAIN_BAND_6 = "INTERNAL_CALIBRATION"
- CORRECTION_GAIN_BAND_7 = "CPF"
- CORRECTION_BIAS_BAND_1 = "CPF"
- CORRECTION_BIAS_BAND_2 = "CPF"
- CORRECTION_BIAS_BAND_3 = "CPF"
- CORRECTION_BIAS_BAND_4 = "CPF"
- CORRECTION_BIAS_BAND_5 = "CPF"
- CORRECTION_BIAS_BAND_6 = "CPF"
- CORRECTION_BIAS_BAND_7 = "CPF"
+ CORRECTION_METHOD_GAIN_BAND1 = "CPF"
+ CORRECTION_METHOD_GAIN_BAND2 = "CPF"
+ CORRECTION_METHOD_GAIN_BAND3 = "CPF"
+ CORRECTION_METHOD_GAIN_BAND4 = "CPF"
+ CORRECTION_METHOD_GAIN_BAND5 = "CPF"
+ CORRECTION_METHOD_GAIN_BAND6 = "IC"
+ CORRECTION_METHOD_GAIN_BAND7 = "CPF"
+ CORRECTION_METHOD_BIAS = "CPF"
+ SUN_AZIMUTH = 127.9202713
+ SUN_ELEVATION = 63.0740593
+ OUTPUT_FORMAT = "GEOTIFF"
   END_GROUP = PRODUCT_PARAMETERS
- GROUP = RADIOMETRIC_RESCALING
- RADIANCE_MULT_BAND_1 = 0.766
- RADIANCE_MULT_BAND_2 = 1.448
- RADIANCE_MULT_BAND_3 = 1.044
- RADIANCE_MULT_BAND_4 = 0.876
- RADIANCE_MULT_BAND_5 = 0.120
- RADIANCE_MULT_BAND_6 = 0.055
- RADIANCE_MULT_BAND_7 = 0.066
- RADIANCE_ADD_BAND_1 = -2.286
- RADIANCE_ADD_BAND_2 = -4.288
- RADIANCE_ADD_BAND_3 = -2.214
- RADIANCE_ADD_BAND_4 = -2.386
- RADIANCE_ADD_BAND_5 = -0.490
- RADIANCE_ADD_BAND_6 = 1.182
- RADIANCE_ADD_BAND_7 = -0.216
- END_GROUP = RADIOMETRIC_RESCALING
+ GROUP = CORRECTIONS_APPLIED
+ STRIPING_BAND1 = "NONE"
+ STRIPING_BAND2 = "NONE"
+ STRIPING_BAND3 = "NONE"
+ STRIPING_BAND4 = "NONE"
+ STRIPING_BAND5 = "NONE"
+ STRIPING_BAND6 = "NONE"
+ STRIPING_BAND7 = "NONE"
+ BANDING = "N"
+ COHERENT_NOISE = "N"
+ MEMORY_EFFECT = "Y"
+ SCAN_CORRELATED_SHIFT = "Y"
+ INOPERABLE_DETECTORS = "N"
+ DROPPED_LINES = "N"
+ END_GROUP = CORRECTIONS_APPLIED
   GROUP = PROJECTION_PARAMETERS
+ REFERENCE_DATUM = "WGS84"
+ REFERENCE_ELLIPSOID = "WGS84"
+ GRID_CELL_SIZE_THM = 30.000
+ GRID_CELL_SIZE_REF = 30.000
+ ORIENTATION = "NUP"
+ RESAMPLING_OPTION = "CC"
     MAP_PROJECTION = "UTM"
- DATUM = "WGS84"
- ELLIPSOID = "WGS84"
- UTM_ZONE = 35
- GRID_CELL_SIZE_REFLECTIVE = 30.00
- GRID_CELL_SIZE_THERMAL = 30.00
- ORIENTATION = "NORTH_UP"
- RESAMPLING_OPTION = "CUBIC_CONVOLUTION"
- MAP_PROJECTION_L0RA = "NA"
   END_GROUP = PROJECTION_PARAMETERS
+ GROUP = UTM_PARAMETERS
+ ZONE_NUMBER = 35
+ END_GROUP = UTM_PARAMETERS
+ GROUP = GEOMETRIC_VERIFY
+ UL_QUAD_RMSE = 1.00
+ UR_QUAD_RMSE = 1.00
+ LR_QUAD_RMSE = 1.00
+ LL_QUAD_RMSE = 1.00
+ SCENE_RMSE = 1.00
+ END_GROUP = GEOMETRIC_VERIFY
END_GROUP = L1_METADATA_FILE
END
+
\ No newline at end of file
-->%---

Thank you, Nikos

(attachments)

LT51810332008141MTI00_MTL_vs_MTLold.diff (68.8 KB)