[GRASS5] WWW-GRASS follow up

I've tryied to ask Federico some questions and I mentioned MapServer. I was going to
try things similar to the excellent work of Federico but stopped when I found MapServer
a ready to use system.
However there are issues...
Map Server is currently fully supporting shape file, and partially SDE (no query support).
It is not a full functional GIS, GRASS offers more support on this side.
It is going to support OGR, and consequently OGCI with query support.
Lacks a rdbms interface...
On the other hand MapServer has a powerful CGI interface based on templates and
configuration files 'dramatically' similar to the one written by Federico.
I'm still confused (and now more than before) on what to use for web interfaces.
I'm also 'afraid' that while free-source world is going to support HTML based web
applications, vendors' world is moving toward XML and content abstraction approach.
A component interface to follow this trend would be welcome!

Regards

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

roberto micarelli wrote:

I've tryied to ask Federico some questions and I mentioned MapServer. I was going to
try things similar to the excellent work of Federico but stopped when I found MapServer
a ready to use system.
However there are issues...
Map Server is currently fully supporting shape file, and partially SDE (no query support).
It is not a full functional GIS, GRASS offers more support on this side.
It is going to support OGR, and consequently OGCI with query support.
Lacks a rdbms interface...
On the other hand MapServer has a powerful CGI interface based on templates and
configuration files 'dramatically' similar to the one written by Federico.
I'm still confused (and now more than before) on what to use for web interfaces.
I'm also 'afraid' that while free-source world is going to support HTML based web
applications, vendors' world is moving toward XML and content abstraction approach.
A component interface to follow this trend would be welcome!

Roberto,

A few notes that might (or might not) complicate things.

o It is my hope to integrate GDAL into MapServer, and GDAL now has good
   support for GRASS rasters via libgrass.

o I would like to see libgrass eventually include the vector API, at which
   time I would see extending OGR to include GRASS vector support also putting
   it within MapServer.

o A significant issue for mapserver applications is _optimized_ access to
   vector data. It isn't clear to me that the current GRASS vector structures
   support optimized spatial or attributes queries, correct me if I am wrong.
   There has been talk of overhauling the GRASS vector API, and I think that
   would be a good opportunity to build it as an optimized datastore for vector
   information with fast spatial and attribute queries. It might be that
   the new vector API supports a variety of possible datastores including
   a simple file based datastore (fitting nicely within the current GRASS
   database heirarchy) and a more sophisticated RDBMS based datastore which
   could be in PostgreSQL, MySQL or even a commercial DB.

In the meantime I think you could do well for fairly static applications with
a wholesale conversion of your datasets to Shapefiles for fast and efficient
access.

PS. You mention that OGR lacks a RDBMS interface. I would be pleased to see
an OGR interface to a database, and I don't think it would be that difficult
especially if the geometry was just kept in "OGC Well Known Binary" format.
I see alot of potential for synergy between projects like MapServer, FMaps,
OSSIM and interoperability technologies like OGR, OGDI and OGC Simple Features
if we (the GIS free software community) could "get behind" a database methology
for storing vector data.

Best Regards,

---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerda@home.com
light and sound - activate the windows | http://members.home.com/warmerda
and watch the world go round - Rush | Geospatial Programmer for Rent

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

Frank Warmerdam wrote:

roberto micarelli wrote:
>
> [speech...]

A few notes that might (or might not) complicate things.

o It is my hope to integrate GDAL into MapServer, and GDAL now has good
   support for GRASS rasters via libgrass.

o I would like to see libgrass eventually include the vector API, at which
   time I would see extending OGR to include GRASS vector support also putting
   it within MapServer.

o A significant issue for mapserver applications is _optimized_ access to
   vector data. It isn't clear to me that the current GRASS vector structures
   support optimized spatial or attributes queries, correct me if I am wrong.
   There has been talk of overhauling the GRASS vector API, and I think that
   would be a good opportunity to build it as an optimized datastore for vector
   information with fast spatial and attribute queries. It might be that
   the new vector API supports a variety of possible datastores including
   a simple file based datastore (fitting nicely within the current GRASS
   database heirarchy) and a more sophisticated RDBMS based datastore which
   could be in PostgreSQL, MySQL or even a commercial DB.

In the meantime I think you could do well for fairly static applications with
a wholesale conversion of your datasets to Shapefiles for fast and efficient
access.

PS. You mention that OGR lacks a RDBMS interface. I would be pleased to see
an OGR interface to a database, and I don't think it would be that difficult
especially if the geometry was just kept in "OGC Well Known Binary" format.
I see alot of potential for synergy between projects like MapServer, FMaps,
OSSIM and interoperability technologies like OGR, OGDI and OGC Simple Features
if we (the GIS free software community) could "get behind" a database methology
for storing vector data.

Best Regards,

All this discussion makes me think at the INFO database of Arc/Info, or at
the Oracle Geo Spatial Engine. Unfortunately these are hundred-thousand-dollars
systems and I'm a bunch-of-dollars guy :frowning: But this is the free-source engine :slight_smile:
I'm working in loading PostgreSQL with data read from e00 tables (I've cloned
m.in.e00 and changed to my needs). I mean, not only attributes but also geometry
arcs loaded in POLYGON columns, indexed as RTREE (Guttman's quadratic split algorithm).
I was surprised of good time responses when clipping 300.000 and more arcs
using a bounding box. This could partially match your proposal of a rdbms
driver for OGR, but I prefer not to write twice something already existing and
need to share experience with someone who eventually tried the same.
It is obvious the advantage of having huge datastores with both geometry and
attributes relationally linked together.
Also I'm trying to keep integration with my network-analiser who, at the
contrary, prefers a dedicated file format.

So, as you can see, I would love to implement all this in a stable flavour
but I feel that more discussions and design ideas must emerge before
forging code. I totally agree with you in that synergy can allow us to get
the best from everyone.

Regards

P.S.: maybe I should enter PostgreSQL support list and ask there for geometry
fields' use experiences and future development plannings. Make me know your
opinion.

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

On Sat, Oct 14, 2000 at 10:43:22AM -0500, Frank Warmerdam wrote:

A few notes that might (or might not) complicate things.

o It is my hope to integrate GDAL into MapServer, and GDAL now has
good support for GRASS rasters via libgrass.

o I would like to see libgrass eventually include the vector API, at
which time I would see extending OGR to include GRASS vector support
also putting it within MapServer.

o A significant issue for mapserver applications is _optimized_
access to vector data. It isn't clear to me that the current GRASS
vector structures support optimized spatial or attributes queries,
correct me if I am wrong. There has been talk of overhauling the
GRASS vector API, and I think that would be a good opportunity to
build it as an optimized datastore for vector information with fast
spatial and attribute queries. It might be that the new vector API
supports a variety of possible datastores including a simple file
based datastore (fitting nicely within the current GRASS database
heirarchy) and a more sophisticated RDBMS based datastore which could
be in PostgreSQL, MySQL or even a commercial DB.

GRASS doesn't have a very good concept of vector attributes to begin
with. IMHO, GRASS needs a serious overhaul of it's vector format and
libraries. It need both spatial and attribute indexing (presuming we
get some decent attribute handling).

But, as far as publishing GRASS data to something like PostgreSQL,
ORACLE or Informix (where there's a conception of geometry), I think it
would be pretty straight forward for whoever wants to step up to the
plate. PostgreSQL has a little bit of a problem with the size of
tuples, but it does do a sort of indexing based on the bounding box of
POINT, PATH and POLYGON types (doesn't know about holes though).

In the meantime I think you could do well for fairly static
applications with a wholesale conversion of your datasets to
Shapefiles for fast and efficient access.

PS. You mention that OGR lacks a RDBMS interface. I would be pleased
to see an OGR interface to a database, and I don't think it would be
that difficult especially if the geometry was just kept in "OGC Well
Known Binary" format. I see alot of potential for synergy between
projects like MapServer, FMaps, OSSIM and interoperability
technologies like OGR, OGDI and OGC Simple Features if we (the GIS
free software community) could "get behind" a database methology for
storing vector data.

The only comment I would make is the WKB is based on a complete object
geometry whereas something like GRASS uses an Arc-Node topology.
Translating from Arc-Node to Rings (Polys) is straight forward, going
the other way is more difficult. IMHO, geometries like the WKB and
Shapefile are fine for data that's in a "finished" state, but create
some problems for data that will be edited. I don't think it'd be too
hard to "present" GRASS data to a calling application as WKB to be used
for display or query.

I'd like to look at these issues more closely after we finalize GRASS
5.0. I'm kind of in bug squash mode right now ;-0

--
/bin/sh ~/.signature:
Command not found

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

"Eric G . Miller" wrote:

On Sat, Oct 14, 2000 at 10:43:22AM -0500, Frank Warmerdam wrote:
> A few notes that might (or might not) complicate things.
>
> o It is my hope to integrate GDAL into MapServer, and GDAL now has
> good support for GRASS rasters via libgrass.
>
> o I would like to see libgrass eventually include the vector API, at
> which time I would see extending OGR to include GRASS vector support
> also putting it within MapServer.
>
> o A significant issue for mapserver applications is _optimized_
> access to vector data. It isn't clear to me that the current GRASS
> vector structures support optimized spatial or attributes queries,
> correct me if I am wrong. There has been talk of overhauling the
> GRASS vector API, and I think that would be a good opportunity to
> build it as an optimized datastore for vector information with fast
> spatial and attribute queries. It might be that the new vector API
> supports a variety of possible datastores including a simple file
> based datastore (fitting nicely within the current GRASS database
> heirarchy) and a more sophisticated RDBMS based datastore which could
> be in PostgreSQL, MySQL or even a commercial DB.

GRASS doesn't have a very good concept of vector attributes to begin
with. IMHO, GRASS needs a serious overhaul of it's vector format and
libraries. It need both spatial and attribute indexing (presuming we
get some decent attribute handling).

The new dig5 format will have a flat database structure, where the
attributes are indices referenced to another database which holds the
data, probably dbmi. But I think this is about the last hack that this
library can take. At the very least the vector engine needs
re-implemented to remove a lot of flotsam and jetsom left over from
previous versions, and maybe a redesign would be in order, though there
are features especially of spatial data handling that I think are well
represented in GRASS and should be preserved and enhanced. See my
remarks below.

But, as far as publishing GRASS data to something like PostgreSQL,
ORACLE or Informix (where there's a conception of geometry), I think it
would be pretty straight forward for whoever wants to step up to the
plate. PostgreSQL has a little bit of a problem with the size of
tuples, but it does do a sort of indexing based on the bounding box of
POINT, PATH and POLYGON types (doesn't know about holes though).

Well as per my remarks above, it might not be a difficult thing in
principle, but the postgres (eg) connectivity would have to be
integarted more closely with the core API. The existing postgres support
works at module level, it allows data to be dumped to postgres and
queried, and after writing the connection is essentially lost.

> In the meantime I think you could do well for fairly static
> applications with a wholesale conversion of your datasets to
> Shapefiles for fast and efficient access.
>
> PS. You mention that OGR lacks a RDBMS interface. I would be pleased
> to see an OGR interface to a database, and I don't think it would be
> that difficult especially if the geometry was just kept in "OGC Well
> Known Binary" format. I see alot of potential for synergy between
> projects like MapServer, FMaps, OSSIM and interoperability
> technologies like OGR, OGDI and OGC Simple Features if we (the GIS
> free software community) could "get behind" a database methology for
> storing vector data.

The only comment I would make is the WKB is based on a complete object
geometry whereas something like GRASS uses an Arc-Node topology.
Translating from Arc-Node to Rings (Polys) is straight forward, going
the other way is more difficult. IMHO, geometries like the WKB and
Shapefile are fine for data that's in a "finished" state, but create
some problems for data that will be edited. I don't think it'd be too
hard to "present" GRASS data to a calling application as WKB to be used
for display or query.

On the whole I would agree with this. The SDTS standard includes a
description of the type of data that is the minimum structure for
various vector formats, presumably because they contain the minimum
required structures for any reliable reconstruction of topological
vector map data. Those are for intercahnge, but I think they adequately
describe resident data as well. Suitable geospatial vector formats are
networks, chains, 2d-manifolds and the like, not rings.

GRASS uses something like this. The `Level 2' format created by
including a dig_plus is built with v.build (probably should be a library
API call instead of a module). You might wonder if the vector data
should be built up from the simplest primitives, ie. points, organised
into chains, connected at nodes and so on, with all the more complex
structures built up from indexed forms of the primitive. A structure
like this (really a sort of graph or network) is built during import in
v.in.shape. But I found it is very resource hungry, so the existing form
where a series of points is the primitive, providing `chains' or edges
or polylines, with implicit nodes, is a good compromise: no topological
integrity is lost while it is faster and leaner. This is a description
of what is AKA `arc-node' format.

Ring->arc-node is not only more difficult, but I believe also more
intrinsically corruptible. When you try to edit whole ring structures in
situ, it doesn't matter how you code, because of the double digitising
over any line, the process is inherently unstable and leads to corrupt
data. Vector maps are like tech drawings, they're blueprints that need
to be built up in detail from the simpler primitives. The holistic
approach is what you take if you want a vector graphic image. However
because on-the-fly generation of rings from arc-node is so fast and
easy, it might be possible to include a rendered form of maps in a kind
of cache for display of finished maps - maybe querying, displaying,
labelling, manipulating purely as a graphic image - but not editing and
not as the definitive form of the map data.

David

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

Hi Roberto,

On Sat, Oct 14, 2000 at 09:59:27PM +0200, roberto micarelli wrote:
<snip>

I'm working in loading PostgreSQL with data read from e00 tables (I've cloned
m.in.e00 and changed to my needs). I mean, not only attributes but also geometry

as you have already modified m.in.e00 to read attributes into
PostgreSQL, could you add this to the CVS? Attribute storage into
PG is quite important. Geometry storage is another point, I am just
thinking of attributes. It would be great to have such feature in
GRASS 5.0stable (comparing to v.in.shape). Do you think that's feasible
for the next days (as GRASS 5.0stable shall be published very soon)?

Thanks

Markus Neteler

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

On dom, 15 ott 2000, Markus Neteler wrote:

Hi Roberto,

On Sat, Oct 14, 2000 at 09:59:27PM +0200, roberto micarelli wrote:
<snip>
> I'm working in loading PostgreSQL with data read from e00 tables (I've cloned
> m.in.e00 and changed to my needs). I mean, not only attributes but also geometry
as you have already modified m.in.e00 to read attributes into
PostgreSQL, could you add this to the CVS? Attribute storage into
PG is quite important. Geometry storage is another point, I am just
thinking of attributes. It would be great to have such feature in
GRASS 5.0stable (comparing to v.in.shape). Do you think that's feasible
for the next days (as GRASS 5.0stable shall be published very soon)?

Thanks

Markus Neteler

'very soon' means that I can give a different program to load INFO tables in PostgreSQL.
Leaving the geometry import task to m.in.e00
In fact I've modified m.in.e00 very hard, loosing the digs load functinality and making it
generate a PG table for each INFO table. This task is done in a more general way than
the original and non standard tables can be read and generated too.
I've done it for test purposes of Postgres with my datastores. If you think it can be enought
I can proceed.

Roberto

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

Hello all.

"Eric G . Miller" wrote:

PostgreSQL has a little bit of a problem with the size of
tuples

Just a minor note that the Postgres team are working on eliminating this
tuple size limitation. The last I heard it should be gone by Postgres
7.1, the next major release, but they said they could not guarantee that
the tuple size limitation would be removed for geometric types. However,
this was some months ago, so this could have changed.

--
Sincerely,

Jazzman (a.k.a. Justin Hickey) e-mail: jhickey@hpcc.nectec.or.th
High Performance Computing Center
National Electronics and Computer Technology Center (NECTEC)
Bangkok, Thailand

People who think they know everything are very irritating to those
of us who do. ---Anonymous

Jazz and Trek Rule!!!

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

On Sun, Oct 15, 2000 at 07:29:33PM +0200, Roberto Micarelli wrote:

On dom, 15 ott 2000, Markus Neteler wrote:
> Hi Roberto,
>
> On Sat, Oct 14, 2000 at 09:59:27PM +0200, roberto micarelli wrote:
> <snip>
> > I'm working in loading PostgreSQL with data read from e00 tables (I've cloned
> > m.in.e00 and changed to my needs). I mean, not only attributes but also geometry
> as you have already modified m.in.e00 to read attributes into
> PostgreSQL, could you add this to the CVS? Attribute storage into
> PG is quite important. Geometry storage is another point, I am just
> thinking of attributes. It would be great to have such feature in
> GRASS 5.0stable (comparing to v.in.shape). Do you think that's feasible
> for the next days (as GRASS 5.0stable shall be published very soon)?
>
> Thanks
>
> Markus Neteler
>

'very soon' means that I can give a different program to load INFO tables in PostgreSQL.
Leaving the geometry import task to m.in.e00

Hi Roberto,

maybe Michel Wurtz is willing to integrate your program into
m.in.e00 (hi Michel!).

Yours

Markus

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'