MI SDTS format - Dragon Slayers Wanted (fwd)

GRASS GRASS user
1

  Anyone with the time, knowledge and interest to work on a data translator?
This would be a tremendous benefit to the GRASS community as well as those
folks still stuck in MapInfo and the like.

Rich

Dr. Richard B. Shepard, President

                       Applied Ecosystem Services, Inc. (TM)
              Making environmentally-responsible mining happen. (SM)
                       --------------------------------
            2404 SW 22nd Street | Troutdale, OR 97060-1247 | U.S.A.
+ 1 503-667-4517 (voice) | + 1 503-667-8863 (fax) | rshepard@appl-ecosys.com

---------- Forwarded message ----------
From: Bill Thoen <bthoen@ctmap.com>

Essgee19@aol.com wrote:

I've spent considerable time in processing this data format with MapInfo
and (sorry to offend anyone with the following) ARCInfo / ArcView . It has
been my experience that although there are work-arounds for getting the
map / graphic aspects of the data into MapInfo, you will not get the ALL
of the proper corresponding attribute data UNLESS you use ARCInfo running
on NT / Unix platforms.

Well, naturally. The USGS developed that format with one of their goals
being that it work with their installed base of Arc/INFO systems. (Of
course, the official goal was to produce a open, public format to facilitate
data transfer between all federal government agencies.) But in either case
ESRI had a strong interest to get in early to ensure that one of their major
customers (the USGS) could use this format on Arc/INFO software. The USGS
has a strong interest in ensuring that the software they use will be able to
handle the SDTS format, so they turned to ESRI for assistance. Working
together, the USGS gets help on the development, and ESRI gets ahead of the
GIS pack. The format is now pretty much available to the public; there's
lots of free USGS data out there for online snarfing; the USGS can read and
write the format using their leased software, and ESRI leads the pack in
SDTS support. All goals met on paper, but what about the spirit of the
thing? We who don't have Arc/INFO available are left holding the short end
of the stick.

This alliance was no secret collusion. The process has been open right from
the start to anyone who wanted to get involved. ESRI may have been dragged
into it early on by USGS requests, but what would tell one of your major
customers? That you're not interested? So ESRI gets the jump on everyone
here, and it takes competitors and third-party developers two years to bring
other translators to market, and these all cost hundreds of dollars. Some
still don't have a decent translator. SDTS is a very difficult format to
fully support, and IMHO, it's reach exceeds its grasp, but still there ought
to be a good PUBLIC translator out there so that the PUBLIC and all the
other non-federal govt agencies can get some use out of it. As it is, your
tax dollars went into this development and for whatever reasons, if you
still can't use it unless you lease Arc/INFO for $20 grand, then you are
just paying your tithe to ESRI.

If we must have this format rammed down our throats, I would like to see it
so easy to use that federal state, county and municipal governments would
prefer to use it when offering their data to the public than E00 (a
proprietary format) or MapInfo TAB (another proprietary format), or anything
that requires expensive, proprietary software. Now that Sol Katz is no
longer with us, who do we have out there who will step up and provide public
support for this public format? If you think you might be interested, take
a look at this data format beastie at http://mcmcweb.er.usgs.gov/sdts/.

- Bill Thoen

2

Rich Shepard wrote:

  Anyone with the time, knowledge and interest to work on a data translator?
This would be a tremendous benefit to the GRASS community as well as those
folks still stuck in MapInfo and the like.

...

Bill Thoen says:
If we must have this format rammed down our throats, I would like to see it
so easy to use that federal state, county and municipal governments would
prefer to use it when offering their data to the public than E00 (a
proprietary format) or MapInfo TAB (another proprietary format), or anything
that requires expensive, proprietary software. Now that Sol Katz is no
longer with us, who do we have out there who will step up and provide public
support for this public format? If you think you might be interested, take
a look at this data format beastie at http://mcmcweb.er.usgs.gov/sdts/.

Rich,

First, isn't there already some SDTS support code for GRASS somewhere? I
believe that CERL wrote a grass to SDTS translator, and one of the SDTS
prototype datasets on the USGS site is generated from GRASS.

Second, I am quite familiar with SDTS, and would be willing to assist in
writing a GRASS module to import it. I have an SDTS reading library already
implemented, and a prototype SDTS to Shape program available. See my
SDTS project page for more information.

  http://gdal.velocet.ca/projects/sdts/index.html

Third, from having done the v.in.shape module for GRASS, I learned that I don't
quite get the GRASS data model for vectors. In particular, what the
appropriate way is to associate multiple attribute fields with vectors. If
there is someone who needs SDTS data, and is willing to work with me on it,
that would help alot.

Best regards,

---------------------------------------+--------------------------------------
I set the clouds in motion - turned up | Frank Warmerdam, warmerda@home.com
light and sound - activate the windows | http://members.home.com/warmerda
and watch the world go round - Rush | Geospatial Programmer for Rent

3

Rich, there's already a SDTS program in GRASS in /src.contrib/SDTS
directory. Would that help?

Frank, feel free to give me a shout about SDTS sometime. I've worked
a bit with it.

Bruce

Frank Warmerdam wrote:

Rich Shepard wrote:
>
> Anyone with the time, knowledge and interest to work on a data translator?
> This would be a tremendous benefit to the GRASS community as well as those
> folks still stuck in MapInfo and the like.
...
> Bill Thoen says:
> If we must have this format rammed down our throats, I would like to see it
> so easy to use that federal state, county and municipal governments would
> prefer to use it when offering their data to the public than E00 (a
> proprietary format) or MapInfo TAB (another proprietary format), or anything
> that requires expensive, proprietary software. Now that Sol Katz is no
> longer with us, who do we have out there who will step up and provide public
> support for this public format? If you think you might be interested, take
> a look at this data format beastie at http://mcmcweb.er.usgs.gov/sdts/.

Rich,

First, isn't there already some SDTS support code for GRASS somewhere? I
believe that CERL wrote a grass to SDTS translator, and one of the SDTS
prototype datasets on the USGS site is generated from GRASS.

Second, I am quite familiar with SDTS, and would be willing to assist in
writing a GRASS module to import it. I have an SDTS reading library already
implemented, and a prototype SDTS to Shape program available. See my
SDTS project page for more information.

  http://gdal.velocet.ca/projects/sdts/index.html

Third, from having done the v.in.shape module for GRASS, I learned that I don't
quite get the GRASS data model for vectors. In particular, what the
appropriate way is to associate multiple attribute fields with vectors. If
there is someone who needs SDTS data, and is willing to work with me on it,
that would help alot.

Best regards,

---------------------------------------+--------------------------------------
I set the clouds in motion - turned up | Frank Warmerdam, warmerda@home.com
light and sound - activate the windows | http://members.home.com/warmerda
and watch the world go round - Rush | Geospatial Programmer for Rent

4

Frank Warmerdam wrote:

Rich Shepard wrote:
>
> Anyone with the time, knowledge and interest to work on a data translator?
> This would be a tremendous benefit to the GRASS community as well as those
> folks still stuck in MapInfo and the like.

[...]

Third, from having done the v.in.shape module for GRASS, I learned that I don't
quite get the GRASS data model for vectors. In particular, what the
appropriate way is to associate multiple attribute fields with vectors. If
there is someone who needs SDTS data, and is willing to work with me on it,
that would help alot.

Hello Frank and Rich,

I'm not familiar with SDTS, but have worked with the people who developped
EDIGeO, which is a french equivalent of SDTS. I presume that like any other
Norm, this kind of data exchange format must take in account every little
trick of every proprietary format, so it becomes a huge and unimplementable
mess that is of course not widely used (high price and complexity of
available implementations). I speak mainly about EDIGeO. I trust more
SDTS because I believe US people are more pragmatic than French ones :slight_smile:

Besides, I think the real problem with Grass vector format is the lack of
a good database interface for storing and managing attributes data.

I'm actually finishing (and hope to publish it for Christmas) a new
version of m.in.e00 for Grass5.0 (some bug corrected, float
grid format supported, and compressed format directly read).
My only problem is "how to have a better attribute support than
to spread all of them in a lot of cat files ?". Gdabse, MySQL or
Postgress : what is the best for Grass community ?
Is there an "official" answer from Baylor people ?

I think that all of us who develop data translator programs must
have the same approach and work together for the multiple attribute
fields problem, and synchronize with Baylor development team.

Any answer or suggestion ?

BTW, I plan to write a v.out.e00 function : does this interest
someone ? (apart Markus, who suggested this to me some... hmmm...
months ago :slight_smile:

Best regards,

--
Michel Wurtz ENGEES - CEREG
                1, quai Koch - BP 1039, F-67070 STRASBOURG cedex
                Tel: +33 03.88.24.82.45 Fax: +33 03.88.37.04.97

5

Hi all!

On Thu, 16 Dec 1999, Michel Wurtz - ENGEES/CEREG wrote:

My only problem is "how to have a better attribute support than
to spread all of them in a lot of cat files ?". Gdabse, MySQL or
Postgress : what is the best for Grass community ?
Is there an "official" answer from Baylor people ?

Well, I think several people already implemented a PostgreSQL
interface. It is time to coordinate these DBMS developments (even I
would be glad to have someone else than me doing it)!

I think that all of us who develop data translator programs must
have the same approach and work together for the multiple attribute
fields problem, and synchronize with Baylor development team.

I could offer a WWW-home for development as I already did for
Postgrass (which is not complete...):
http://www.geog.uni-hannover.de/grass/projects/postgrass/

BTW, I plan to write a v.out.e00 function : does this interest
someone ? (apart Markus, who suggested this to me some... hmmm...
months ago :slight_smile:

Of course. Or v.out.shape. v.out.shape is already written by
BLACKLAND-GRASS, but I am still waiting for their contribution
to GRASS.

I think it is not a secret: A new GRASS vector format was already
designed by CERL (vector 5 API). I post the address here for
all to get ideas from it:
http://www.roadkill.com/~dpgerdes/vector/api.html

[the email address from David Gerdes on that page is not valid any more.]

Best regards

Markus Neteler

6

On Thu, 16 Dec 1999, Michel Wurtz - ENGEES/CEREG wrote:

... this kind of data exchange format must take in account every little
trick of every proprietary format, so it becomes a huge and unimplementable
mess that is of course not widely used (high price and complexity of
available implementations).

Michel,

  I followed the development of SDTS for the first few years, then gave up
the effort. However, I believe the problem with it is less trying to
accommodate every proprietary data format, and more trying to be a "one size
fits all" data store.

  The fundamental idea of SDTS was (is?) that it should hold every different
type of spatial data produced or used by US government agencies. Not just
natural world data, but engineered/built data, too. So, here's a data
structure which will hold geographic objects, their locatations and their
attributes for airports, dams, highways, office buildings, census data about
people and soils, vegetation, rivers and species distributions. That's like
having one suit of clothes for every occasion; it just can't be easily done.

  Then, just to make the situation more obtuse, pick a complex, binary
format for everything with multiple headers for each datum type, a complex
API, and poorly written instructions. Finally, change things every now and
then. Then release it as the only format for all spatial data. Ugh!

Besides, I think the real problem with Grass vector format is the lack of
a good database interface for storing and managing attributes data.

  That's the problem for raster data, too. For example, cells could have
attributes for soil, depth-to-bedrock, slope, vegetation, aspect,
erodability, and so on. In my opinion, a GIS cannot do meaningful analyses
or modeling without a solid, relational database in which to store
attributes. Mapping, yes; spatial analyses/modeling, no.

My only problem is "how to have a better attribute support than to spread
all of them in a lot of cat files ?". Gdabse, MySQL or Postgress : what
is the best for Grass community ? Is there an "official" answer from
Baylor people ?

  Let me go out on a limb, here: PostgreSQL. It has native, graphic data
types (and the 8K per tuple limit will be greatly increased in version 7),
had a working interface to GRASS in earler versions of both, and has a team
in Italy working on upgrading that interface.

  Without provoking a flame war on which dbms is "best", I am strongly
supportive of "officially" supporting postgres. There are a number of
reasons, but that's not relevant here and now. For what it's worth, I need
that GRASS/postgres interface before I can move my projects to GRASS. It's
critical for the work we want to do.

  I'm still available to work on that interface.

Rich
  
Dr. Richard B. Shepard, President

                       Applied Ecosystem Services, Inc. (TM)
              Making environmentally-responsible mining happen. (SM)
                       --------------------------------
            2404 SW 22nd Street | Troutdale, OR 97060-1247 | U.S.A.
+ 1 503-667-4517 (voice) | + 1 503-667-8863 (fax) | rshepard@appl-ecosys.com

7

Hi, Michel
Thanks for your ongoing contributions. In my opinion, m.out.e00 would be
useful to a sizeable subset of GRASS users (including myself and my
cohorts here at CAST), but what we really all need are good shapefile
import/export capabilities, as, at least here in the US, shapefiles have
become almost a defacto standard for vector data (yuk!). v.in.shape is a
great start (thanks Frank!), but has some noteable limitations dealing
with polygon data (unless this has been remedied since I last looked at
it). Since my end goal is usually raster-based analysis in GRASS, the
majority of vector data that I'm interested in importing is
polygon-based.

I wish I were better able to assist in this area, but translators are
currently out of my league!

Best regards,
  -Malcolm

Malcolm D. Williamson malcolm@cast.uark.edu
Center for Advanced Spatial Technologies Voice: 501-575-2734
12 Ozark Hall Fax: 501-575-5218
University of Arkansas, Fayetteville AR 72701
http://www.cast.uark.edu/

Michel Wurtz - ENGEES/CEREG wrote:

Hello Frank and Rich,

I'm not familiar with SDTS, but have worked with the people who developped
EDIGeO, which is a french equivalent of SDTS. I presume that like any other
Norm, this kind of data exchange format must take in account every little
trick of every proprietary format, so it becomes a huge and unimplementable
mess that is of course not widely used (high price and complexity of
available implementations). I speak mainly about EDIGeO. I trust more
SDTS because I believe US people are more pragmatic than French ones :slight_smile:

Besides, I think the real problem with Grass vector format is the lack of
a good database interface for storing and managing attributes data.

I'm actually finishing (and hope to publish it for Christmas) a new
version of m.in.e00 for Grass5.0 (some bug corrected, float
grid format supported, and compressed format directly read).
My only problem is "how to have a better attribute support than
to spread all of them in a lot of cat files ?". Gdabse, MySQL or
Postgress : what is the best for Grass community ?
Is there an "official" answer from Baylor people ?

I think that all of us who develop data translator programs must
have the same approach and work together for the multiple attribute
fields problem, and synchronize with Baylor development team.

Any answer or suggestion ?

BTW, I plan to write a v.out.e00 function : does this interest
someone ? (apart Markus, who suggested this to me some... hmmm...
months ago :slight_smile:

Best regards,

--
Michel Wurtz ENGEES - CEREG
                1, quai Koch - BP 1039, F-67070 STRASBOURG cedex
                Tel: +33 03.88.24.82.45 Fax: +33 03.88.37.04.97

8

Rich Shepard wrote:
(lines deleted)

  That's the problem for raster data, too. For example, cells could have
attributes for soil, depth-to-bedrock, slope, vegetation, aspect,
erodability, and so on. In my opinion, a GIS cannot do meaningful analyses
or modeling without a solid, relational database in which to store
attributes. Mapping, yes; spatial analyses/modeling, no.

(more lines deleted)

Rich,
I think you're approaching GRASS with a little too strong of a
relational database mindset, at least on the raster side of things. The
GRASS raster format already supports multiple "fields" or attributes for
a cell... they're called reclasses. Remember, all r.reclass does is
create a lookup table. It is possible to have thousands of attributes
for the same cells, and still have efficient data recall. I have found
this model to be every bit as functional as the linked database model
used in some competing raster packages, and I would dare to say that
there are quite a few other serious modelers out in GRASS land that
would agree.

Now for vector data, with it's significantly more complex topology,
etc., an rdbms data model makes a lot of sense to me.
--
Best regards,
  -Malcolm

Malcolm D. Williamson malcolm@cast.uark.edu
Center for Advanced Spatial Technologies Voice: 501-575-2734
12 Ozark Hall Fax: 501-575-5218
University of Arkansas, Fayetteville AR 72701
http://www.cast.uark.edu/