[GRASS5] gdal 1.3.1 and Terra ASTER hdf files

cmbarton · November 4, 2005, 12:29am

I work with the GRASS GIS project and haven’t been able to import and georeference Terra ASTER hdf files since gdal version 1.2.6. I recently compiled and installed the most recent 1.3.1 (as of last week) and it still produces errors (see below). As an update, I just tried r.in.gdal from GRASS 6.1 cvs and it will not import the ASTER files either. First it did not recognize the projection, so I set the override projection flag. Then it won’t recognize a band number or import all bands (default).

So, back to my original question, what has changed since 1.2.6 that keeps me from importing ASTER’s? Could this be fixed?

Thanks much
Michael Barton

Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
Arizona State University
Tempe, AZ 85287-2402
USA

voice: 480-965-6262; fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

On Thu, Nov 03, 2005 at 09:02:00AM -0700, Michael Barton wrote:

I tried to import Terra ASTER hdf files using the latest gdalwarp and my
r.in.aster script last week, after compiling the latest version (1.3.1) of
gdal. It still gives an error that gdal cannot read the projection info in
the ASTER hdf’s. This has been an issue since 1.2.7 (version 1.2.6 still
works fine on my Mac).

I haven’t tried r.in.gdal, but suspect that I’d have similar problems
(please correct me if wrong!).

Is there a fix or workaround to this? All ASTERs are in WGS84, UTM
projection (the zone varies of course).

Michael

Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
Arizona State University
Tempe, AZ 85287-2402
USA

voice: 480-965-6262; fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton

From: Hamish hamish_nospam@yahoo.com
Date: Thu, 3 Nov 2005 14:44:44 +1300
To: Jonathan Greenberg jgreenberg@arc.nasa.gov
Cc: GRASSLIST@baylor.edu
Subject: [GRASSLIST:8849] Re: HDF files?

Is there any way to get HDF files into GRASS? I saw there is an
r.in.hdf command, but Grass 6.0.1 on MacOS X is throwing and error
when I try it: command not found

Use r.in.gdal, but GDAL must be built using the --with-hdf4 switch.

http://www.remotesensing.org/gdal/frmt_hdf4.html

try “gdal_translate --formats” and check for:

HDF4 (ro): Hierarchical Data Format Release 4
HDF4Image (rw+): HDF4 Dataset

or just “gdalinfo ” and see if it understands the file(s).

“r.in.gdal -f in=dummy out=dummy” will also list supported GDAL formats.
(often there will be about 50 of them)

Hamish

Carl_Anderson3 · November 4, 2005, 2:43am

Michael Barton wrote:

I work with the GRASS GIS project and haven’t been able to import and georeference Terra ASTER hdf files since gdal version 1.2.6. I recently compiled and installed the most recent 1.3.1 (as of last week) and it still produces errors (see below). As an update, I just tried r.in.gdal from GRASS 6.1 cvs and it will not import the ASTER files either. First it did not recognize the projection, so I set the override projection flag. Then it won’t recognize a band number or import all bands (default).

So, back to my original question, what has changed since 1.2.6 that keeps me from importing ASTER’s? Could this be fixed?

I'll try to relate what I understand. The GDAL 1.2.6 driver was limited and did not do a very good job with all of the information in HDF files. The old HDF sub driver was changed to newer driver to improve the overall quality. In 1.2.9? (certainly by 1.3.0) the driver was working well. The downside is that you cannot directly "read" projected data from a HDF and must gdalwarp L1A and L2A HDF files to a specific projection to get access to projected data. If you look at any ASTER HDF file you will see projected data mixed in with sensor calibration samples and other non geophysical data. I suspect that is the reason that a ASTER HDF appears unprojected through the GDAL drivers.

FWTools 1.0.0a4 == GDAL 1.3.1.0, FWTools 1.0.0a4, released 2005/10/15

/usr/local/FWTools-linux-1.0.0a4/bin_safe/gdalinfo AST_L1B_00306152005162314_06242005150609.hdf
...
PGEVERSION=06.00R00
PROCESSINGLEVELID=1B
MAPPROJECTIONNAME=Universal Transverse Mercator
Subdatasets:
SUBDATASET_1_NAME=HDF4_EOS:EOS_SWATH:"AST_L1B_00306152005162314_06242005150609.hdf":VNIR_Swath:ImageData1
SUBDATASET_1_DESC=[4200x4980] ImageData1 (8-bit unsigned integer)
SUBDATASET_2_NAME=HDF4_EOS:EOS_SWATH:"AST_L1B_00306152005162314_06242005150609.hdf":VNIR_Swath:ImageData2
SUBDATASET_2_DESC=[4200x4980] ImageData2 (8-bit unsigned integer)
...
Corner Coordinates:
Upper Left ( 0.0, 0.0)
Lower Left ( 0.0, 512.0)
Upper Right ( 512.0, 0.0)
Lower Right ( 512.0, 512.0)
Center ( 256.0, 256.0)

/usr/local/FWTools-linux-1.0.0a4/bin_safe/gdalwarp -t_srs EPSG:26916 HDF4_EOS:EOS_SWATH:"AST_L1B_00306152005162314_06242005150609.hdf":VNIR_Swath:ImageData1 -tps b.tif

/usr/local/FWTools-linux-1.0.0a4/bin_safe/gdalinfo b.tif
...
Upper Left ( 714191.497, 3718692.860) ( 84d41'31.09"W, 33d35'11.27"N)
...

You will need to warp each band from the HDF file before importing.

C.

Thanks much
Michael Barton

______________________________
Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
Arizona State University
Tempe, AZ 85287-2402
USA

voice: 480-965-6262; fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton <http://www.public.asu.edu/~cmbarton>

On Thu, Nov 03, 2005 at 09:02:00AM -0700, Michael Barton wrote:

I tried to import Terra ASTER hdf files using the latest gdalwarp and my
r.in.aster script last week, after compiling the latest version

(1.3.1) of

gdal. It still gives an error that gdal cannot read the projection

info in

the ASTER hdf's. This has been an issue since 1.2.7 (version 1.2.6 still
works fine on my Mac).

I haven't tried r.in.gdal, but suspect that I'd have similar problems
(please correct me if wrong!).

Is there a fix or workaround to this? All ASTERs are in WGS84, UTM
projection (the zone varies of course).

Michael
______________________________
Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
Arizona State University
Tempe, AZ 85287-2402
USA

voice: 480-965-6262; fax: 480-965-7671
www: _http://www.public.asu.edu/~cmbarton

<http://www.public.asu.edu/~cmbarton>
_>

> From: Hamish <_hamish_nospam@yahoo.com_>
> Date: Thu, 3 Nov 2005 14:44:44 +1300
> To: Jonathan Greenberg <_jgreenberg@arc.nasa.gov_>
> Cc: <_GRASSLIST@baylor.edu_>
> Subject: [GRASSLIST:8849] Re: HDF files?
>
>> Is there any way to get HDF files into GRASS? I saw there is an
>> r.in.hdf command, but Grass 6.0.1 on MacOS X is throwing and error
>> when I try it: command not found
>
> Use r.in.gdal, but GDAL must be built using the --with-hdf4 switch.
>
> _http://www.remotesensing.org/gdal/frmt_hdf4.html

_> >

> try "gdal_translate --formats" and check for:
>
> HDF4 (ro): Hierarchical Data Format Release 4
> HDF4Image (rw+): HDF4 Dataset
>
> or just "gdalinfo <dataset>" and see if it understands the file(s).
>
> "r.in.gdal -f in=dummy out=dummy" will also list supported GDAL

formats.

> (often there will be about 50 of them)
>
> Hamish

cmbarton · November 4, 2005, 4:34am

Thanks for the information Carl.

From: Carl Anderson <carl.anderson@vadose.org>
Date: Thu, 03 Nov 2005 21:43:11 -0500
To: Michael Barton <michael.barton@asu.edu>
Cc: <gdal-dev@lists.maptools.org>, Hamish <hamish_nospam@yahoo.com>, GRASS
Developer's List <grass5@grass.itc.it>
Subject: Re: [GRASS5] gdal 1.3.1 and Terra ASTER hdf files

I'll try to relate what I understand. The GDAL 1.2.6 driver was limited
and did not do a very good job with all of the information in HDF files.
The old HDF sub driver was changed to newer driver to improve the
overall quality. In 1.2.9? (certainly by 1.3.0) the driver was working
well. The downside is that you cannot directly "read" projected data
from a HDF and must gdalwarp L1A and L2A HDF files to a specific
projection to get access to projected data.

Snip snip

You will need to warp each band from the HDF file before importing.

So how do I do this? The r.in.aster script that I wrote for GRASS awhile
back essentially runs gdalwarp on each image band (one at a time) as
follows...

gdalwarp -t_srs "`g.proj -jf`" $srcfile $tempfile

It gets the target projection data from GRASS and doesn't specify the source
(i.e., ASTER) projection data. This seems to be what you are recommending.
However, gdalwarp still chokes on the ASTER source projection information.

Any suggestions? Running gdalinfo on an ASTER reveals projection
information. Is there some way to grep that info into a format that gdalwarp
would accept? Or is it a larger problem (i.e., that it ignores GCP's or
something).

Michael
__________________________________________
Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
Arizona State University
Tempe, AZ 85287-2402

phone: 480-965-6213
fax: 480-965-7671
www: http://www.public.asu.edu/~cmbarton