[GRASS-dev] [GRASS-user] i.landsat.toar unable to read metadata file

Nikos:

I tested the modified landsat_met.c code, provided by E. Jorge Tizado. It
works! Yet, it fails somewhere else, e.g.:

--%<---
# get toar
i.landsat.toar input_prefix=${BASE} output_prefix=${BASE}_toar
metfile=${BASE}_MTL.txt

ERROR: Unknown production date (defined by 'product_date')
--->%--

It seems there are several modifications to the MET content structure.
Fortunately (?) two MET files are provided *currently* (at least I have seen
this in several L[MTE][457] scenes.

For example, an LT5 packed scene contains:
..
LT51810332008141MTI00_MTL.txt
LT51810332008141MTI00_MTLold.txt
..

Sorry,

The diff file...

is better read like:

--%<---
--- LT51810332008141MTI00_MTLold.txt 2012-12-05 08:42:36.000000000 +0200
+++ LT51810332008141MTI00_MTL.txt 2012-12-12 18:47:35.393026898 +0200
@@ -2,140 +2,144 @@
   GROUP = METADATA_FILE_INFO
     ORIGIN = "Image courtesy of the U.S. Geological Survey"
     REQUEST_ID = "0101212018233_00067"
- PRODUCT_CREATION_TIME = 2012-12-05T06:42:36Z
+ LANDSAT_SCENE_ID = "LT51810332008141MTI00"
+ FILE_DATE = 2012-12-05T06:34:04Z
     STATION_ID = "EDC"
- LANDSAT5_XBAND = "1"
- GROUND_STATION = "MTI"
- LPS_PROCESSOR_NUMBER = 0
- DATEHOUR_CONTACT_PERIOD = "0814108"
- SUBINTERVAL_NUMBER = "01"
+ PROCESSING_SOFTWARE_VERSION = "LPGS_12.1.3"
+ DATA_CATEGORY = "NOMINAL"
   END_GROUP = METADATA_FILE_INFO
   GROUP = PRODUCT_METADATA
- PRODUCT_TYPE = "L1T"
+ DATA_TYPE = "L1T"
+ DATA_TYPE_L0RP = "TMR_L0RP"
     ELEVATION_SOURCE = "GLS2000"
- PROCESSING_SOFTWARE = "LPGS_12.1.3"
+ OUTPUT_FORMAT = "GEOTIFF"
     EPHEMERIS_TYPE = "DEFINITIVE"
- SPACECRAFT_ID = "Landsat5"
+ SPACECRAFT_ID = "LANDSAT_5"
     SENSOR_ID = "TM"
     SENSOR_MODE = "BUMPER"
- ACQUISITION_DATE = 2008-05-20
- SCENE_CENTER_SCAN_TIME = 08:40:46.1460440Z
     WRS_PATH = 181
- STARTING_ROW = 33
- ENDING_ROW = 33
- BAND_COMBINATION = "1234567"
- PRODUCT_UL_CORNER_LAT = 39.8635274
- PRODUCT_UL_CORNER_LON = 25.1772902
- PRODUCT_UR_CORNER_LAT = 39.8736219
- PRODUCT_UR_CORNER_LON = 27.9904391
- PRODUCT_LL_CORNER_LAT = 37.9207012
- PRODUCT_LL_CORNER_LON = 25.2262979
- PRODUCT_LR_CORNER_LAT = 37.9301230
- PRODUCT_LR_CORNER_LON = 27.9637968
- PRODUCT_UL_CORNER_MAPX = 344100.000
- PRODUCT_UL_CORNER_MAPY = 4414200.000
- PRODUCT_UR_CORNER_MAPX = 584700.000
- PRODUCT_UR_CORNER_MAPY = 4414200.000
- PRODUCT_LL_CORNER_MAPX = 344100.000
- PRODUCT_LL_CORNER_MAPY = 4198500.000
- PRODUCT_LR_CORNER_MAPX = 584700.000
- PRODUCT_LR_CORNER_MAPY = 4198500.000
- PRODUCT_SAMPLES_REF = 8021
- PRODUCT_LINES_REF = 7191
- PRODUCT_SAMPLES_THM = 8021
- PRODUCT_LINES_THM = 7191
- BAND1_FILE_NAME = "LT51810332008141MTI00_B1.TIF"
- BAND2_FILE_NAME = "LT51810332008141MTI00_B2.TIF"
- BAND3_FILE_NAME = "LT51810332008141MTI00_B3.TIF"
- BAND4_FILE_NAME = "LT51810332008141MTI00_B4.TIF"
- BAND5_FILE_NAME = "LT51810332008141MTI00_B5.TIF"
- BAND6_FILE_NAME = "LT51810332008141MTI00_B6.TIF"
- BAND7_FILE_NAME = "LT51810332008141MTI00_B7.TIF"
- GCP_FILE_NAME = "LT51810332008141MTI00_GCP.txt"
- VER_REPORT_FILE_NAME = "LT51810332008141MTI00_VER.txt"
- VER_BROWSE_FILE_NAME = "LT51810332008141MTI00_VER.jpg"
- METADATA_L1_FILE_NAME = "LT51810332008141MTI00_MTLold.txt"
- CPF_FILE_NAME = "L5CPF20080401_20080630_16"
+ WRS_ROW = 033
+ DATE_ACQUIRED = 2008-05-20
+ SCENE_CENTER_TIME = 08:40:46.1460440Z
+ CORNER_UL_LAT_PRODUCT = 39.86353
+ CORNER_UL_LON_PRODUCT = 25.17729
+ CORNER_UR_LAT_PRODUCT = 39.87362
+ CORNER_UR_LON_PRODUCT = 27.99044
+ CORNER_LL_LAT_PRODUCT = 37.92070
+ CORNER_LL_LON_PRODUCT = 25.22630
+ CORNER_LR_LAT_PRODUCT = 37.93012
+ CORNER_LR_LON_PRODUCT = 27.96380
+ CORNER_UL_PROJECTION_X_PRODUCT = 344100.000
+ CORNER_UL_PROJECTION_Y_PRODUCT = 4414200.000
+ CORNER_UR_PROJECTION_X_PRODUCT = 584700.000
+ CORNER_UR_PROJECTION_Y_PRODUCT = 4414200.000
+ CORNER_LL_PROJECTION_X_PRODUCT = 344100.000
+ CORNER_LL_PROJECTION_Y_PRODUCT = 4198500.000
+ CORNER_LR_PROJECTION_X_PRODUCT = 584700.000
+ CORNER_LR_PROJECTION_Y_PRODUCT = 4198500.000
+ REFLECTIVE_LINES = 7191
+ REFLECTIVE_SAMPLES = 8021
+ THERMAL_LINES = 7191
+ THERMAL_SAMPLES = 8021
+ FILE_NAME_BAND_1 = "LT51810332008141MTI00_B1.TIF"
+ FILE_NAME_BAND_2 = "LT51810332008141MTI00_B2.TIF"
+ FILE_NAME_BAND_3 = "LT51810332008141MTI00_B3.TIF"
+ FILE_NAME_BAND_4 = "LT51810332008141MTI00_B4.TIF"
+ FILE_NAME_BAND_5 = "LT51810332008141MTI00_B5.TIF"
+ FILE_NAME_BAND_6 = "LT51810332008141MTI00_B6.TIF"
+ FILE_NAME_BAND_7 = "LT51810332008141MTI00_B7.TIF"
+ GROUND_CONTROL_POINT_FILE_NAME = "LT51810332008141MTI00_GCP.txt"
+ REPORT_VERIFY_FILE_NAME = "LT51810332008141MTI00_VER.txt"
+ BROWSE_VERIFY_FILE_NAME = "LT51810332008141MTI00_VER.jpg"
+ METADATA_FILE_NAME = "LT51810332008141MTI00_MTL.txt"
+ CPF_NAME = "L5CPF20080401_20080630.16"
   END_GROUP = PRODUCT_METADATA
+ GROUP = IMAGE_ATTRIBUTES
+ CLOUD_COVER = 10.00
+ IMAGE_QUALITY = 9
+ SUN_AZIMUTH = 127.92027132
+ SUN_ELEVATION = 63.07405928
+ GROUND_CONTROL_POINTS_MODEL = 1
+ GEOMETRIC_RMSE_MODEL = 3.741
+ GEOMETRIC_RMSE_MODEL_Y = 2.933
+ GEOMETRIC_RMSE_MODEL_X = 2.323
+ GROUND_CONTROL_POINTS_VERIFY = 1265
+ GEOMETRIC_RMSE_VERIFY = 0.160
+ END_GROUP = IMAGE_ATTRIBUTES
   GROUP = MIN_MAX_RADIANCE
- LMAX_BAND1 = 193.000
- LMIN_BAND1 = -1.520
- LMAX_BAND2 = 365.000
- LMIN_BAND2 = -2.840
- LMAX_BAND3 = 264.000
- LMIN_BAND3 = -1.170
- LMAX_BAND4 = 221.000
- LMIN_BAND4 = -1.510
- LMAX_BAND5 = 30.200
- LMIN_BAND5 = -0.370
- LMAX_BAND6 = 15.303
- LMIN_BAND6 = 1.238
- LMAX_BAND7 = 16.500
- LMIN_BAND7 = -0.150
+ RADIANCE_MAXIMUM_BAND_1 = 193.000
+ RADIANCE_MINIMUM_BAND_1 = -1.520
+ RADIANCE_MAXIMUM_BAND_2 = 365.000
+ RADIANCE_MINIMUM_BAND_2 = -2.840
+ RADIANCE_MAXIMUM_BAND_3 = 264.000
+ RADIANCE_MINIMUM_BAND_3 = -1.170
+ RADIANCE_MAXIMUM_BAND_4 = 221.000
+ RADIANCE_MINIMUM_BAND_4 = -1.510
+ RADIANCE_MAXIMUM_BAND_5 = 30.200
+ RADIANCE_MINIMUM_BAND_5 = -0.370
+ RADIANCE_MAXIMUM_BAND_6 = 15.303
+ RADIANCE_MINIMUM_BAND_6 = 1.238
+ RADIANCE_MAXIMUM_BAND_7 = 16.500
+ RADIANCE_MINIMUM_BAND_7 = -0.150
   END_GROUP = MIN_MAX_RADIANCE
   GROUP = MIN_MAX_PIXEL_VALUE
- QCALMAX_BAND1 = 255.0
- QCALMIN_BAND1 = 1.0
- QCALMAX_BAND2 = 255.0
- QCALMIN_BAND2 = 1.0
- QCALMAX_BAND3 = 255.0
- QCALMIN_BAND3 = 1.0
- QCALMAX_BAND4 = 255.0
- QCALMIN_BAND4 = 1.0
- QCALMAX_BAND5 = 255.0
- QCALMIN_BAND5 = 1.0
- QCALMAX_BAND6 = 255.0
- QCALMIN_BAND6 = 1.0
- QCALMAX_BAND7 = 255.0
- QCALMIN_BAND7 = 1.0
+ QUANTIZE_CAL_MAX_BAND_1 = 255
+ QUANTIZE_CAL_MIN_BAND_1 = 1
+ QUANTIZE_CAL_MAX_BAND_2 = 255
+ QUANTIZE_CAL_MIN_BAND_2 = 1
+ QUANTIZE_CAL_MAX_BAND_3 = 255
+ QUANTIZE_CAL_MIN_BAND_3 = 1
+ QUANTIZE_CAL_MAX_BAND_4 = 255
+ QUANTIZE_CAL_MIN_BAND_4 = 1
+ QUANTIZE_CAL_MAX_BAND_5 = 255
+ QUANTIZE_CAL_MIN_BAND_5 = 1
+ QUANTIZE_CAL_MAX_BAND_6 = 255
+ QUANTIZE_CAL_MIN_BAND_6 = 1
+ QUANTIZE_CAL_MAX_BAND_7 = 255
+ QUANTIZE_CAL_MIN_BAND_7 = 1
   END_GROUP = MIN_MAX_PIXEL_VALUE
   GROUP = PRODUCT_PARAMETERS
- CORRECTION_METHOD_GAIN_BAND1 = "CPF"
- CORRECTION_METHOD_GAIN_BAND2 = "CPF"
- CORRECTION_METHOD_GAIN_BAND3 = "CPF"
- CORRECTION_METHOD_GAIN_BAND4 = "CPF"
- CORRECTION_METHOD_GAIN_BAND5 = "CPF"
- CORRECTION_METHOD_GAIN_BAND6 = "IC"
- CORRECTION_METHOD_GAIN_BAND7 = "CPF"
- CORRECTION_METHOD_BIAS = "CPF"
- SUN_AZIMUTH = 127.9202713
- SUN_ELEVATION = 63.0740593
- OUTPUT_FORMAT = "GEOTIFF"
+ CORRECTION_GAIN_BAND_1 = "CPF"
+ CORRECTION_GAIN_BAND_2 = "CPF"
+ CORRECTION_GAIN_BAND_3 = "CPF"
+ CORRECTION_GAIN_BAND_4 = "CPF"
+ CORRECTION_GAIN_BAND_5 = "CPF"
+ CORRECTION_GAIN_BAND_6 = "INTERNAL_CALIBRATION"
+ CORRECTION_GAIN_BAND_7 = "CPF"
+ CORRECTION_BIAS_BAND_1 = "CPF"
+ CORRECTION_BIAS_BAND_2 = "CPF"
+ CORRECTION_BIAS_BAND_3 = "CPF"
+ CORRECTION_BIAS_BAND_4 = "CPF"
+ CORRECTION_BIAS_BAND_5 = "CPF"
+ CORRECTION_BIAS_BAND_6 = "CPF"
+ CORRECTION_BIAS_BAND_7 = "CPF"
   END_GROUP = PRODUCT_PARAMETERS
- GROUP = CORRECTIONS_APPLIED
- STRIPING_BAND1 = "NONE"
- STRIPING_BAND2 = "NONE"
- STRIPING_BAND3 = "NONE"
- STRIPING_BAND4 = "NONE"
- STRIPING_BAND5 = "NONE"
- STRIPING_BAND6 = "NONE"
- STRIPING_BAND7 = "NONE"
- BANDING = "N"
- COHERENT_NOISE = "N"
- MEMORY_EFFECT = "Y"
- SCAN_CORRELATED_SHIFT = "Y"
- INOPERABLE_DETECTORS = "N"
- DROPPED_LINES = "N"
- END_GROUP = CORRECTIONS_APPLIED
+ GROUP = RADIOMETRIC_RESCALING
+ RADIANCE_MULT_BAND_1 = 0.766
+ RADIANCE_MULT_BAND_2 = 1.448
+ RADIANCE_MULT_BAND_3 = 1.044
+ RADIANCE_MULT_BAND_4 = 0.876
+ RADIANCE_MULT_BAND_5 = 0.120
+ RADIANCE_MULT_BAND_6 = 0.055
+ RADIANCE_MULT_BAND_7 = 0.066
+ RADIANCE_ADD_BAND_1 = -2.286
+ RADIANCE_ADD_BAND_2 = -4.288
+ RADIANCE_ADD_BAND_3 = -2.214
+ RADIANCE_ADD_BAND_4 = -2.386
+ RADIANCE_ADD_BAND_5 = -0.490
+ RADIANCE_ADD_BAND_6 = 1.182
+ RADIANCE_ADD_BAND_7 = -0.216
+ END_GROUP = RADIOMETRIC_RESCALING
   GROUP = PROJECTION_PARAMETERS
- REFERENCE_DATUM = "WGS84"
- REFERENCE_ELLIPSOID = "WGS84"
- GRID_CELL_SIZE_THM = 30.000
- GRID_CELL_SIZE_REF = 30.000
- ORIENTATION = "NUP"
- RESAMPLING_OPTION = "CC"
     MAP_PROJECTION = "UTM"
+ DATUM = "WGS84"
+ ELLIPSOID = "WGS84"
+ UTM_ZONE = 35
+ GRID_CELL_SIZE_REFLECTIVE = 30.00
+ GRID_CELL_SIZE_THERMAL = 30.00
+ ORIENTATION = "NORTH_UP"
+ RESAMPLING_OPTION = "CUBIC_CONVOLUTION"
+ MAP_PROJECTION_L0RA = "NA"
   END_GROUP = PROJECTION_PARAMETERS
- GROUP = UTM_PARAMETERS
- ZONE_NUMBER = 35
- END_GROUP = UTM_PARAMETERS
- GROUP = GEOMETRIC_VERIFY
- UL_QUAD_RMSE = 1.00
- UR_QUAD_RMSE = 1.00
- LR_QUAD_RMSE = 1.00
- LL_QUAD_RMSE = 1.00
- SCENE_RMSE = 1.00
- END_GROUP = GEOMETRIC_VERIFY
END_GROUP = L1_METADATA_FILE
END
-
-->%---

After checking the *current* MTL.txt files along with the landsat_met.c code,
it seems that there is one unified MTL.txt file for all sensors now, right?

Nikos

(attachments)

LT51810332008141MTI00_MTLold_vs_MTL.diff (68.8 KB)

Nikos:

> It seems there are several modifications to the MET content structure.
> Fortunately (?) two MET files are provided *currently* (at least I have
> seen this in several L[MTE][457] scenes.

..

I tried to compare variable definitions in the landsat_met.c file with the diff
"TLold.txt_VS_MTL.txt" file (metadata that correspond to a LT5 scene). The
differences I see are the following:

- SPACECRAFT_ID: was Landsat5, is: LANDSAT_5

- SENSOR_ID: *this is the same*

- ACQUISITION_DATE: renamed (?) to "DATE_ACQUIRED"

- CREATION_TIME: was (?) PRODUCT_CREATION_TIME, is: FILE_DATE

- SUN_ELEVATION: was not existing (!), is: same

- BAND%_GAIN: nowhere to be found as is, was (?):
CORRECTION_METHOD_GAIN_BAND%, is: CORRECTION_GAIN_BAND_%

- LMAX_BAND%: renamed to RADIANCE_MAXIMUM_BAND_%

- LMIN_BAND%: same as above, renamed to RADIANCE_MINIMUM_BAND_%

- QCALMAX_BAND%: renamed to QUANTIZE_CAL_MAX_BAND_%

- QCALMIN_BAND: same as above, renamed to QUANTIZE_CAL_MIN_BAND_%

Is this enough to get i.landsat.toar working?
Thanks, Nikos

Try to add an additional strcmp() case where ACQUISITION_DATE is
changed to DATE_ACQUIRED

sorry I am very busy these few coming days...

On 13 December 2012 12:42, Nikos Alexandris <nik@nikosalexandris.net> wrote:

Nikos:

> It seems there are several modifications to the MET content structure.
> Fortunately (?) two MET files are provided *currently* (at least I have
> seen this in several L[MTE][457] scenes.

..

I tried to compare variable definitions in the landsat_met.c file with the diff
"TLold.txt_VS_MTL.txt" file (metadata that correspond to a LT5 scene). The
differences I see are the following:

- SPACECRAFT_ID: was Landsat5, is: LANDSAT_5

- SENSOR_ID: *this is the same*

- ACQUISITION_DATE: renamed (?) to "DATE_ACQUIRED"

- CREATION_TIME: was (?) PRODUCT_CREATION_TIME, is: FILE_DATE

- SUN_ELEVATION: was not existing (!), is: same

- BAND%_GAIN: nowhere to be found as is, was (?):
CORRECTION_METHOD_GAIN_BAND%, is: CORRECTION_GAIN_BAND_%

- LMAX_BAND%: renamed to RADIANCE_MAXIMUM_BAND_%

- LMIN_BAND%: same as above, renamed to RADIANCE_MINIMUM_BAND_%

- QCALMAX_BAND%: renamed to QUANTIZE_CAL_MAX_BAND_%

- QCALMIN_BAND: same as above, renamed to QUANTIZE_CAL_MIN_BAND_%

Is this enough to get i.landsat.toar working?
Thanks, Nikos
_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

--
Yann Chemin

(re-structured the last post, cc-ed András B, E. Jorge T -- someone is not on
the list?)

Nikos:

>> > It seems there are several modifications to the MET content structure.
>> > Fortunately (?) two MET files are provided *currently* (at least I have
>> > seen this in several L[MTE][457] scenes.

..

Nikos:

> I tried to compare variable definitions in the landsat_met.c file with the
> diff "TLold.txt_VS_MTL.txt" file (metadata that correspond to a LT5
> scene). The differences I see are the following:

> - SPACECRAFT_ID: was Landsat5, is: LANDSAT_5
> - SENSOR_ID: *this is the same*

> - ACQUISITION_DATE: renamed (?) to "DATE_ACQUIRED"
> - CREATION_TIME: was (?) PRODUCT_CREATION_TIME, is: FILE_DATE

> - SUN_ELEVATION: was not existing (!), is: same
> - BAND%_GAIN: nowhere to be found as is, was (?):
> CORRECTION_METHOD_GAIN_BAND%, is: CORRECTION_GAIN_BAND_%

> - LMAX_BAND%: renamed to RADIANCE_MAXIMUM_BAND_%
> - LMIN_BAND%: same as above, renamed to RADIANCE_MINIMUM_BAND_%

> - QCALMAX_BAND%: renamed to QUANTIZE_CAL_MAX_BAND_%
> - QCALMIN_BAND: same as above, renamed to QUANTIZE_CAL_MIN_BAND_%

> Is this enough to get i.landsat.toar working?

Yann Chemin:

Try to add an additional strcmp() case where ACQUISITION_DATE is
changed to DATE_ACQUIRED

sorry I am very busy these few coming days...

On Fri, Dec 14, 2012 at 9:13 AM, Nikos Alexandris
<nik@nikosalexandris.net> wrote:

(re-structured the last post, cc-ed András B, E. Jorge T -- someone is not on
the list?)

Nikos:

>> > It seems there are several modifications to the MET content structure.
>> > Fortunately (?) two MET files are provided *currently* (at least I have
>> > seen this in several L[MTE][457] scenes.

..

Nikos:

> I tried to compare variable definitions in the landsat_met.c file with the
> diff "TLold.txt_VS_MTL.txt" file (metadata that correspond to a LT5
> scene). The differences I see are the following:

> - SPACECRAFT_ID: was Landsat5, is: LANDSAT_5
> - SENSOR_ID: *this is the same*

> - ACQUISITION_DATE: renamed (?) to "DATE_ACQUIRED"
> - CREATION_TIME: was (?) PRODUCT_CREATION_TIME, is: FILE_DATE

> - SUN_ELEVATION: was not existing (!), is: same
> - BAND%_GAIN: nowhere to be found as is, was (?):
> CORRECTION_METHOD_GAIN_BAND%, is: CORRECTION_GAIN_BAND_%

> - LMAX_BAND%: renamed to RADIANCE_MAXIMUM_BAND_%
> - LMIN_BAND%: same as above, renamed to RADIANCE_MINIMUM_BAND_%

> - QCALMAX_BAND%: renamed to QUANTIZE_CAL_MAX_BAND_%
> - QCALMIN_BAND: same as above, renamed to QUANTIZE_CAL_MIN_BAND_%

> Is this enough to get i.landsat.toar working?

Yann Chemin:

Try to add an additional strcmp() case where ACQUISITION_DATE is
changed to DATE_ACQUIRED

sorry I am very busy these few coming days...

The original changes I got from Jorge I attach again. Should they be
submitted to SVN?

Markus

(attachments)

landsat_met.c.diff (783 Bytes)

..

Markus:

The original changes I got from Jorge I attach again. Should they be
submitted to SVN?

Jorge did some more changes, I tested and verified that it works with both L5's and L7's along with a new(er) MLT.txt metadata file. Jorge, could you please resend it?

Nikos

ps- due to an hdd failure I lost some stuff (2 weeks of e-mails and data missing from my latest sync because I was non-stop moving these days) so I can't attach the latest i.landsat.toar right now (need to get my hdd fixed first).

Hi all,

I have finally updated the SVN (GRASS 6.4, 6.5 and 7) with the updated
version of i.landsat.toar. Please test.

Best
Markus

Hi all,

I got thermal band fixes from Jorge and applied them to GRASS 6.4.svn
and 6.5.svn.
http://trac.osgeo.org/grass/changeset/55376

They still need to be fwd-ported to GRASS 7.

Markus

Markus Neteler wrote:

I got thermal band fixes from Jorge and applied them to GRASS 6.4.svn

I guess this is the fix related to the post entitled

"i.landsat.toar fails to convert DNs (bands 61 and 62) to temperature for
Landsat 7 scene"

Works in G64 -- tested with an L5 (LT51820362009326MTI00) and an L7
(LE71840312002041SGS00) scene.

Thank you, Nikos :slight_smile:

and 6.5.svn.
http://trac.osgeo.org/grass/changeset/55376

They still need to be fwd-ported to GRASS 7.