[GRASS5] New USGS DEM data format

GRASS GRASS developer
1

  I just learned that the US Geological Survey (USGS) has done two things
less than desirable. First, they no longer provide data for free, especially
the 30-m DEM data. It's only available now at one commercial site, that of
GeoComm (URL avaialable upon request).

  Second, and more importantly, they have changed the format of the
elevation data within the DEM to "decimal meters" which I assume to be
fractional meters. The elevations used to be only in integral feet or
meters. So, on this basis, more accurate DEM data is a Good Thing. But, they
changed the SDTS format to this, and ESRI's ARC/GIS-8 (and Maptitude-5.0)
are the only software that can read and translate the new SDTS format.
GeoComm, the only site at which these data are available, is an ESRI
reseller. Well, leaving out the politics of this arrangement, leaves my
question: can r.in.gdal handle the new format with the "decimal meter"
elevations?

  I've imported a DEM using r.in.gdal, and it didn't complain, but I've no
idea what elevation values it has. Anyone know? Frank?

Thanks,

Rich

Dr. Richard B. Shepard, President

                       Applied Ecosystem Services, Inc. (TM)
            2404 SW 22nd Street | Troutdale, OR 97060-1247 | U.S.A.
+ 1 503-667-4517 (voice) | + 1 503-667-8863 (fax) | rshepard@appl-ecosys.com
                 Making environmentally-responsible mining happen.

2

On Wed, 1 Aug 2001, Rich Shepard wrote:

  Second, and more importantly, they have changed the format of the
elevation data within the DEM to "decimal meters" which I assume to be
fractional meters. The elevations used to be only in integral feet or
meters. So, on this basis, more accurate DEM data is a Good Thing. But, they
changed the SDTS format to this, and ESRI's ARC/GIS-8 (and Maptitude-5.0)
are the only software that can read and translate the new SDTS format.
GeoComm, the only site at which these data are available, is an ESRI
reseller. Well, leaving out the politics of this arrangement, leaves my
question: can r.in.gdal handle the new format with the "decimal meter"
elevations?

  I've imported a DEM using r.in.gdal, and it didn't complain, but I've no
idea what elevation values it has. Anyone know? Frank?

Follow up:

  I just ran r.out.ascii on the DEM. It has apparently rounded (or
truncated?) the elevations to integer values.

  May I put in a request that this be fixed now? I can't imagine that it
requires too much work to change an int to a float.

Thanks,

Rich

Dr. Richard B. Shepard, President

                       Applied Ecosystem Services, Inc. (TM)
            2404 SW 22nd Street | Troutdale, OR 97060-1247 | U.S.A.
+ 1 503-667-4517 (voice) | + 1 503-667-8863 (fax) | rshepard@appl-ecosys.com
                 Making environmentally-responsible mining happen.

3

Rich

your data appear to be all correct - I worked with this type of data just
recently
and they a far supperior to the 30m data (the mine were created by more accurate
interpolation
and due to the higher vertical precission they did not have those nasty 1m steps,
they have
other problems though). Your data are correctly in integer decimeters.
Just divide them by 10. (don't forget the decimal point) in r.mapcalc and you
will get
a floating point DEM in meters with vertical precission in dm. Unless there is
some additional
problem that I don't know about this should do it.

Helena

Shepard wrote:

On Wed, 1 Aug 2001, Rich Shepard wrote:

> Second, and more importantly, they have changed the format of the
> elevation data within the DEM to "decimal meters" which I assume to be
> fractional meters. The elevations used to be only in integral feet or
> meters. So, on this basis, more accurate DEM data is a Good Thing. But, they
> changed the SDTS format to this, and ESRI's ARC/GIS-8 (and Maptitude-5.0)
> are the only software that can read and translate the new SDTS format.
> GeoComm, the only site at which these data are available, is an ESRI
> reseller. Well, leaving out the politics of this arrangement, leaves my
> question: can r.in.gdal handle the new format with the "decimal meter"
> elevations?
>
> I've imported a DEM using r.in.gdal, and it didn't complain, but I've no
> idea what elevation values it has. Anyone know? Frank?

Follow up:

  I just ran r.out.ascii on the DEM. It has apparently rounded (or
truncated?) the elevations to integer values.

  May I put in a request that this be fixed now? I can't imagine that it
requires too much work to change an int to a float.

Thanks,

Rich

Dr. Richard B. Shepard, President

                       Applied Ecosystem Services, Inc. (TM)
            2404 SW 22nd Street | Troutdale, OR 97060-1247 | U.S.A.
+ 1 503-667-4517 (voice) | + 1 503-667-8863 (fax) | rshepard@appl-ecosys.com
                 Making environmentally-responsible mining happen.

_______________________________________________
grass5 mailing list
grass5@geog.uni-hannover.de
http://www.geog.uni-hannover.de/mailman/listinfo/grass5

4

Rich Shepard wrote:

  I just learned that the US Geological Survey (USGS) has done two things
less than desirable. First, they no longer provide data for free, especially
the 30-m DEM data. It's only available now at one commercial site, that of
GeoComm (URL avaialable upon request).

Where did your information come from? They haven't done that yet, and I
don't see any evidence on their site that they intend to ever do that.

The web site at http://edcwww.cr.usgs.gov/webglis appears to offer all
of the same elevation data that it always did, including the 2
arc-second (30-minute) DEMs. I would check with Earthexplorer
(http://earthexplorer.usgs.gov) as well, but I can't. They keep
threatening to take webglis down and force all traffic through
earthexplorer, which would be unfortunate. I have never gotten
EarthExplorer to work right.

  Second, and more importantly, they have changed the format of the
elevation data within the DEM to "decimal meters" which I assume to be
fractional meters. The elevations used to be only in integral feet or
meters. So, on this basis, more accurate DEM data is a Good Thing.

I would be surprised if they added any accuracy by changing the file
format. In fact, I would be surprise if they changed the file format on
anything they already released in an older format. They put out new
digital map data all the time (especially in the 7.5 minute series).
Perhaps they are using a new format for the newly released data?

Roger Miller

5

On Wed, 1 Aug 2001, Roger S. Miller wrote:

Where did your information come from? They haven't done that yet, and I
don't see any evidence on their site that they intend to ever do that.

Roger,

  I just forwarded Bill's message to the GRASS-5 list.

They keep threatening to take webglis down and force all traffic through
earthexplorer, which would be unfortunate. I have never gotten
EarthExplorer to work right.

  Me, neither. I wrote to them and explained that it wouldn't work with
Navigator-4.77, mozilla-0.9.1 or opera-5.0 (identifying itself as MSIE5.0).
Still broken.

I would be surprised if they added any accuracy by changing the file
format. In fact, I would be surprise if they changed the file format on
anything they already released in an older format. They put out new
digital map data all the time (especially in the 7.5 minute series).
Perhaps they are using a new format for the newly released data?

  Read the article. They _did_ change the data and they modified SDTS
format. Strangely enough, only ARC/GIS-8 supports that format. And the GS is
a complete ESRI shop (which is how ESRI got to be as big as they are).

  Politics hits the GIS world, too. Welcome to my world. :slight_smile:

Rich

Dr. Richard B. Shepard, President

                       Applied Ecosystem Services, Inc. (TM)
            2404 SW 22nd Street | Troutdale, OR 97060-1247 | U.S.A.
+ 1 503-667-4517 (voice) | + 1 503-667-8863 (fax) | rshepard@appl-ecosys.com
                 Making environmentally-responsible mining happen.

6

Rich Shepard wrote:

On Wed, 1 Aug 2001, Roger S. Miller wrote:

> Where did your information come from? They haven't done that yet, and I
> don't see any evidence on their site that they intend to ever do that.

Roger,

  I just forwarded Bill's message to the GRASS-5 list.

Thanks, I read it. I guess they made the transition to the
GeoCommunities servers pretty seamlessly. Or maybe the step from
webglis to earthexplorer *is* the shift to GeoCommunities, in which case
it isn't working very well.

> They keep threatening to take webglis down and force all traffic through
> earthexplorer, which would be unfortunate. I have never gotten
> EarthExplorer to work right.

  Me, neither. I wrote to them and explained that it wouldn't work with
Navigator-4.77, mozilla-0.9.1 or opera-5.0 (identifying itself as MSIE5.0).
Still broken.

I've written them a couple times about it. Most recently today. They
have been announcing that shift for a long time now and just moving the
date back. Hopefully they will get it right before they force the
shift. So far, earthexplorer looks like a massive mistake.

> I would be surprised if they added any accuracy by changing the file
> format. In fact, I would be surprise if they changed the file format on
> anything they already released in an older format. They put out new
> digital map data all the time (especially in the 7.5 minute series).
> Perhaps they are using a new format for the newly released data?

  Read the article. They _did_ change the data and they modified SDTS
format. Strangely enough, only ARC/GIS-8 supports that format. And the GS is
a complete ESRI shop (which is how ESRI got to be as big as they are).

Actually, as I read the article it appears that the format change
they're talking about isn't a change *in* the SDTS format, but the (now
old news) change *to* the SDTS format. The article is talking about the
differences between the old DEM format (which I think was mostly phased
out more than a year ago) and the SDTS format.

They underestimate the number of applications that read SDTS. The SDTS
standard was published quite a while ago. Surfer started importing SDTS
files in version 6, which was released in 1997. I could have sworn that
GRASS used to have at least a v.in.sdts, but now I can't find it. I
must be going crazy. Some programs were available from government
agencies for reading SDTS files almost as soon as the standard was
published.

Roger Miller

7

On Wed, 1 Aug 2001, Roger S. Miller wrote:

Actually, as I read the article it appears that the format change
they're talking about isn't a change *in* the SDTS format, but the (now
old news) change *to* the SDTS format. The article is talking about the
differences between the old DEM format (which I think was mostly phased
out more than a year ago) and the SDTS format.

Roger,

  I don't know. My friend, Bill Thoen, told me that the GS made a subtle
change to the SDTS format. Since I've not followed that for almost a decade,
I cannot comment with any intelligence.

  Yes, they are moving all data from DLG to SDTS, but otherwise I'm just a
bystander.

Rich

Dr. Richard B. Shepard, President

                       Applied Ecosystem Services, Inc. (TM)
            2404 SW 22nd Street | Troutdale, OR 97060-1247 | U.S.A.
+ 1 503-667-4517 (voice) | + 1 503-667-8863 (fax) | rshepard@appl-ecosys.com
                 Making environmentally-responsible mining happen.

8

On Wed, Aug 01, 2001 at 08:55:11PM -0600, Roger S. Miller wrote:
[snip]

> Read the article. They _did_ change the data and they modified SDTS
> format. Strangely enough, only ARC/GIS-8 supports that format. And the GS is
> a complete ESRI shop (which is how ESRI got to be as big as they are).

Actually, as I read the article it appears that the format change
they're talking about isn't a change *in* the SDTS format, but the (now
old news) change *to* the SDTS format. The article is talking about the
differences between the old DEM format (which I think was mostly phased
out more than a year ago) and the SDTS format.

I couldn't find anything about changes to the SDTS format since 1999 at
the SDTS website (last standard extension for raster profiles). I think
the format change might be meters --> decimeters thing??? If they made
other significant changes that are not conforming to the standard, then
they really shouldn't call it SDTS anymore!

They underestimate the number of applications that read SDTS. The SDTS
standard was published quite a while ago. Surfer started importing SDTS
files in version 6, which was released in 1997. I could have sworn that
GRASS used to have at least a v.in.sdts, but now I can't find it. I
must be going crazy. Some programs were available from government
agencies for reading SDTS files almost as soon as the standard was
published.

v.in.sdts is in GRASS 5 and works (I've used it) -- It's in
src.contrib/SDTS/ and uses the FIPS123 library (for ISO 8211 format
reading). As noted, r.in.gdal seems to work okay (though it didn't read
the projection info for me when I tested it yesterday). r.in.gdal
doesn't give you access to all the metadata info like v.in.sdts will --
but there is m.sdts.read which is a generic ISO 8211 to text reader
(works well).

--
Eric G. Miller <egm2@jps.net>

9

Rich Shepard wrote:

  I just learned that the US Geological Survey (USGS) has done two things
less than desirable. First, they no longer provide data for free, especially
the 30-m DEM data. It's only available now at one commercial site, that of
GeoComm (URL avaialable upon request).

Uhh ? Becoming European ? :wink:

  Second, and more importantly, they have changed the format of the
elevation data within the DEM to "decimal meters" which I assume to be
fractional meters. The elevations used to be only in integral feet or
meters. So, on this basis, more accurate DEM data is a Good Thing. But, they
changed the SDTS format to this, and ESRI's ARC/GIS-8 (and Maptitude-5.0)
are the only software that can read and translate the new SDTS format.
GeoComm, the only site at which these data are available, is an ESRI
reseller.

There is an article entitled "Recent changes in U.S. mapping data have
upset the GIS community" about this new USGS's strategy at

http://www.directionsmag.com/article.asp?ArticleID=116

--
Michel WURTZ - DIG - Maison de la télédétection
               500, rue J.F. Breton
               34093 MONTPELLIER Cedex 5

10

On Wed, 1 Aug 2001, Eric G. Miller wrote:

I couldn't find anything about changes to the SDTS format since 1999 at
the SDTS website (last standard extension for raster profiles). I think
the format change might be meters --> decimeters thing??? If they made
other significant changes that are not conforming to the standard, then
they really shouldn't call it SDTS anymore!

Eric,

  Yes. I think that you hit it correctly. From the article, I understood
that the change to "decimal meters" broke most commercial/Windows-hosted
software. Why, I've no idea, but there it is.

v.in.sdts is in GRASS 5 and works (I've used it) -- It's in
src.contrib/SDTS/ and uses the FIPS123 library (for ISO 8211 format
reading). As noted, r.in.gdal seems to work okay (though it didn't read
the projection info for me when I tested it yesterday). r.in.gdal
doesn't give you access to all the metadata info like v.in.sdts will --
but there is m.sdts.read which is a generic ISO 8211 to text reader
(works well).

  So, what module do you recommend that I use to import the 9 DEMs from SDTS
format? I've been using r.in.gdal because that worked for me. FWIW, I'm
fascinated by what I'm learning about GRASS by running these modules. The
absolute dependence on having specific files, with specific content, in
specific places is a new experience for me, but I've learned how to handle
it gracefully.

  I've not tried importing vector data from SDTS format; so far, only the
DEMS. And, if you know of 1:24k vector data for transportation, hydrography,
or hypsography for the Western U.S., please do tell me where to find it!

Thanks,

Rich

Dr. Richard B. Shepard, President

                       Applied Ecosystem Services, Inc. (TM)
            2404 SW 22nd Street | Troutdale, OR 97060-1247 | U.S.A.
+ 1 503-667-4517 (voice) | + 1 503-667-8863 (fax) | rshepard@appl-ecosys.com
                 Making environmentally-responsible mining happen.

11

On Thu, 2 Aug 2001, Rich Shepard wrote:

  I've not tried importing vector data from SDTS format; so far, only the
DEMS. And, if you know of 1:24k vector data for transportation, hydrography,
or hypsography for the Western U.S., please do tell me where to find it!

Rich,

Depending on which part of the Western U.S. you're interested in, you may
be able to get it from the USGS sites mentioned previously as large scale
digital line graphs in SDTS format. Coverage varies considerably by state.
I think that New Mexico, for instance, is completely covered. Texas
(which is western in some peoples books) is maybe 50% covered.

Roger Miller

12

Rich (Grass list)

USDA is slowly coming around to placing their data online.
See http://www.nhq.nrcs.usda.gov/land/index/intro.html and click
on the GIS & Data link for more details.

John

----- Original Message -----
From: "Rich Shepard" <rshepard@appl-ecosys.com>
To: "grass5" <grass5@geog.uni-hannover.de>
Cc: "Bill Thoen" <bthoen@ctmap.com>
Sent: Thursday, August 02, 2001 7:35 AM
Subject: Re: [GRASS5] New USGS DEM data format

<snip>
  I've not tried importing vector data from SDTS format; so far, only the
DEMS. And, if you know of 1:24k vector data for transportation, hydrography,
or hypsography for the Western U.S., please do tell me where to find it!

Thanks,

Rich

Dr. Richard B. Shepard, President

                       Applied Ecosystem Services, Inc. (TM)
            2404 SW 22nd Street | Troutdale, OR 97060-1247 | U.S.A.
+ 1 503-667-4517 (voice) | + 1 503-667-8863 (fax) | rshepard@appl-ecosys.com
                 Making environmentally-responsible mining happen.