Hi Chris,
Interesting discussion!
In our mapping language (we call it
CMF, concept mapping file) comparable to schema.xml file we allow
complicate mappings with filters, conditional mappings and things
like that. I suppose is pretty much the same thing you are thinking.Interesting. It could be interesting to try to sync up the work we're
both doing. I don't like everyone reinventing the wheel for mapping
files, but I don't know of anyone else really doing them. Perhaps we
could try to use the same file format between our projects. The
'filters' that we make use of our primarily OGC Filters - their
representation of a where statement.Short (rough) overview here:
http://udig.refractions.net/confluence/display/GOWS/Querying+data+using+'Filter'
and full spec is here:
http://portal.opengeospatial.org/files/?artifact_id=8340 (pdf)
Well, Ok, let's go in parts.
What we are working is in an HTTP-XML protocol to allow dynamic queries to heterogeneous databases and get results in XML compliant to whatever XML schema they have mapped their databases... (puff this may not be the best description of it...). In our community (TDWG, Taxonomic Database Working Group, a very little OGC like standardization group) there are two protocols for doing that, Digir and bioCASe, and now we are merging them into a new one called TAPIR. There will be two implementations of this protocol. The protocol, very much like the WFS protocol, is still under discussion in a wiki page http://ww3.bgbm.org/protocolwiki/FrontPage.
Actually the WFS protocol was reviewed when creating this new protocol because it was considered a possible solution to our problems. If you take a look at our capability operation is very likely to WFS one, and the filtering in a query is also very similar to the one you pointed me (because we also get inspired by it!).
For this protocol we are creating there will be two reference implementations, pywrapper in python, and Digir 2 in Java and RDF based.
We already had an implementation of them in GeoTools, since we're OGC
focused, so they made logical sense to do the mappings (correct me if
I'm wrong here Gabriel). A nice thing about them is that we already
have most of the code written to 'unpack' them, that is if the back end
data format can perform the filter, then we turn it into the SQL (or
whatever query language it might use). So we should be able to get
good performance out of the mappings, since where possible it's passed
to the backend. But the nice thing is that we _can_ do it on anything,
since there's Java code for it all, so Shapefile supports all filters
as well.
Uhm... I do not understand this totally...
You mean that implementing a more intelligent mapping will let you create dynamically SQL statements that you can use for filtering based on the query filter? I feel I may be mixing everything...
In our case we describe the database structure and we map the database to the different schemas that we want to share the data in. When someone do a query the wrapper read this configuration file and figure out the where clause to filter the data in the SQL statement. Then depending on the xml schema that is requested as a response it selects the necessary information from the database to populate an XML instance of this XML schema.
Well, in reality it is a little bit more complicate to give the mapping of the database-schema the necessary information to create correct xml documents... Unfortunately an XML schema can validate many different XML documents that looks completely different...
I am working also
for a European project called Synthesys and I was planning to get in
contact with the INSPIRE project in the near soon when I get more OGC
knowledgeExcellent. If you want some contacts I know at least one good one.
Yes! that would be great... I feel is a shame I am not in contact with them when I am aiming at providing GIS services in a european level...
What you are planning to do with complex mappings looks quite similar
to the problem that we are trying to solve too. As I described we
have a wrapper tool to make possible to connect heterogenous
databases to a network where we can query them using a standard
protocol (TAPIR and BioCASe before), the results from the providers
come in XML following a certain "conceptual schema" (what you call
application schema). Maybe we should have a talk and I will show you
how are we doing things and you may get some ideas...Yeah, that could be good. I don't know how far in to things you are,
but note that a WFS doesn't _have_ to do geometric objects. Like it
could theoretically not return any GML at all, just XML. I don't know
if that's the route to go, but it might weirdly be worthwhile to just
sync your effort up with GeoTools/GeoServer, since we would both
benefit from more users and use cases. But yeah, at the very least it
sounds like we're working on very similar problems, and can learn what
works and what doesn't, even if it doesn't make sense to share at the
coding level.
As I said before when we were discussing our protocol we took a look at available protocols and one of those was WFS. I was not resposible for that and I do not know the exact reason why it was discard (it should be in the protocol review uhmm). But... is it possible with WFS to do paging? I mean... in our case someone retrieve ten "features" and then want to retrieve the next 10...
At the end seems everybody is working on the same problem, the
mapping of relational databases to XML schemas...Yeah, there's a lot out there, and a lot of different approaches, and it
feels silly to roll so much of your own. But many are lacking in
different ways. We're hoping with our approach to eventually be able
to sync up with other more generic things. We looked into using
Hibernate to do the object-relational mapping, but Rob needed to be
able to control the SQL select statements, so you could derive your
mapping from the results of a select statement, which hibernate just
doesn't let you. And we wrote a straight SAX producer to stream out
GML, since at the time there were no good, fast object -> XML producers
that allowed you real flexibility, since we have to deal with arbitrary
Features, of arbitrary complexity, can't define the schema beforehand.
There are solutions now, and I think our next generation
parser/producer makes of JAXB. But yeah, working with Geographic
Information we hit a lot of the big problems early, since the datasets
so quickly get huge.
Yes, in our case this is an issue... we have to be able to page trough datasets because even the fastest wrapper in the world will not be able to retrieve 1 million features full of properties at once... that's why we have paging.
One could thing we are trying to implement, for example, is that the user is able to send the structure, the "complex_type", in an XML schema fragment specifying the result formats he want the data to be responded. There is a two step-mapping going here but I do not want to get long now...
The project web page of one of this implementation is at http://ww3.bgbm.org/tapir and if you want to see a data provider in action you can go to http://ww3.bgbm.org/tapirwrapper/
Best regards,
Javier.
best regards,
Chris
Javier.
On 23/11/2005, at 8:23, Chris Holmes wrote:
Actually, Javier, you're getting into the bleeding edge of
GeoServer,
and you actually understand how WFS and GML _should_ work
perfectly.
Rob has been pushing these types of things for awhile, and just
recently managed to get a lot of the ideas in GeoServer as working
code
with Gabriel. The schema.xml files were not about actually mapping
your GML to different types, they were just kind of surface levelways
of renaming things. The problem of actually getting names to map
back
and forth, to perform filters on them, and more, is much more
complex.
The problem is the code is on a branch, like Gabriel just announced
it
last week, and there will still be bugs, and there's lots of
documentation to write. But we would love to have you as a user
testing it out. Indeed your intended schema is pretty simplecompared
to the complexity of what we're hoping to support.
See the 'proto-documentation' (really just a place holder) here:
http://docs.codehaus.org/display/GEOSDOC/Complex+Feature+Documentation
As for MySQL, if you're using the MySQL module you need to have
your
points stored as Geometries, see:
http://dev.mysql.com/doc/refman/5.0/en/spatial-extensions-in-
mysql.html
(it's available in 4.1 as well). As for improvements from 4.1 to5.0,
I haven't tried out 5.0, but looking at the docs there appears to
be
nothing new, the support overall is pretty poor, I'd recommend
upgrading to PostGIS asap.If you really want to define your MySQL table as just x,y
coordinates,
Rob made a Geometryless module that will work, but we don't
currently
include it with GeoServer. I believe Gabriel upgraded it though in
his
latest work.best regards,
Chris
Quoting Justin Deoliveira <jdeolive@anonymised.com>:
Hi Javier,
So what you need is the wfs to serve up gml that conforms to your
application schema. After you add your data store and feature typeto
geoserver (via the config gui), your feature type will contain
attributes with the default types. You can customize the typeswith
the
feature type editor. There are some docs on how to do this here.http://docs.codehaus.org/display/GEOSDOC/FeatureType+Config
Note the part about changing the type of an attribute to an xml
fragment.Unfortunatley I dont know much about mysql, and even less about
its
spatial extension. However I am sure someone else will be able to
help
with that. And questions about mysql could probably be bestanswered
on
the geotools devel and user lists. I would try repositing those
questions there.If you have any more questions dont hesitate to ask.
Justin
Javier de la Torre wrote:
Hi Justin,
Thanks for your answer.
Some background. I am using MySQL right now as a DataStore (butalso
planning to go for PostGIS). In my database I have a table where
I
have
records of specimens (plants) with information like their name,
who
identified them, when they were collected, etc. Also I have in
the
same
table coordinates of the location where these plants where
collected.
My idea is to create a WFS service to provide the points where
the
specimens where collected together with the other information. We
have
some kind of standards about how all this information must be
described. The idea is to create a GML application schema wherewe
describe the specimens using our own "concepts" (like
CatalogNumber
and
things like that) and the location of the specimen using a
gml:PointPropertyType.If I let Geoserver create automatically the schema for me I get
something like this:----------
<xs:complexType name = "specimens_Type" >
<xs:complexContent>
<xs:extension base = "gml:AbstractFeatureType" >
<xs:sequence>
<xs:element type = "xs:string" minOccurs = "1" name ="SpecimenGUID"
nillable = "true" maxOccurs = "1" />
<xs:element type = "xs:string" minOccurs = "0" name ="ProjectTitle"
nillable = "true" maxOccurs = "1" />
<xs:element type = "xs:string" minOccurs = "0" name ="BiotopeText"
nillable = "true" maxOccurs = "1" />
<xs:element type = "xs:dateTime" minOccurs = "0" name =
"CollectingISODate" nillable = "true" maxOccurs = "1" />
<xs:element type = "xs:string" minOccurs = "0" name ="LocalityText"
nillable = "true" maxOccurs = "1" />
<xs:element type = "xs:string" minOccurs = "0" name ="CountryName"
nillable = "true" maxOccurs = "1" />
<xs:element type = "xs:string" minOccurs = "0" name = "Notes"nillable
= "true" maxOccurs = "1" />
<xs:element type = "xs:int" minOccurs = "0" name ="AltitudeLowerValue"
nillable = "true" maxOccurs = "1" />
<xs:element type = "xs:string" minOccurs = "0" name ="coordinates"
nillable = "true" maxOccurs = "1" />
</xs:sequence>
</xs:extension>
</xs:complexContent>
</xs:complexType>
-------------But I would prefer to get something like:
-----------
<xs:complexType name = "specimens_Type" >
<xs:complexContent>
<xs:extension base = "gml:AbstractFeatureType" >
<xs:sequence>
<xs:element type = "my_ns:GUID" minOccurs = "1" name ="SpecimenGUID"
nillable = "true" maxOccurs = "1" />
<xs:element type = "my_ns:ProjectTitle" minOccurs = "0" name =
"ProjectTitle" nillable = "true" maxOccurs = "1" />
<xs:element type = "my_ns: BiotopeText" minOccurs = "0" name =
"BiotopeText" nillable = "true" maxOccurs = "1" />
<xs:element type = "my_ns:dateTime" minOccurs = "0" name =
"CollectingISODate" nillable = "true" maxOccurs = "1" />
<xs:element type = "my_ns:string" minOccurs = "0" name ="LocalityText"
nillable = "true" maxOccurs = "1" />
<xs:element type = "my_ns:string" minOccurs = "0" name ="CountryName"
nillable = "true" maxOccurs = "1" />
<xs:element type = "my_ns:string" minOccurs = "0" name = "Notes"
nillable = "true" maxOccurs = "1" />
<xs:element type = "my_ns:int" minOccurs = "0" name =
"AltitudeLowerValue" nillable = "true" maxOccurs = "1" />
<xs:element type = "gml:PointPropertyType" minOccurs = "0" name =
"coordinates" nillable = "true" maxOccurs = "1" />
</xs:sequence>
</xs:extension>
</xs:complexContent>
</xs:complexType>
----------------And by the way... how do I have to define the coordinates in my
MySQL
table so that Geoserver understand them as Points in gml? In the
case
of PostGIS is this done automatically?
A final question... I do not find any documentation about the
spatial
capabilities of MySQL 5, do you know if there is any improvement
from
what it was available already in 4.1?
I am new to OGC standards so i am still a little bit confused on
how to
create this GML application schemas so sorry if I am asking no-
sense...
Thanks.
Javier.
On 22/11/2005, at 20:29, Justin Deoliveira wrote:
Hi Javier,
I suspect that the GML writer ignores the information that you
have
supplied in the schema.xml file, and generates the gml from the
actual features / feature types themselves.Since the DescribeFeatureType logic takes place in GeoServer
itself,
it can be smart and delegate to the schema.xml file. However
the
GML
writing logic takes place in the geotools library, and it has
no
notion of this file.
Can you tell us a bit more about how you are trying to customize
your
schema. Perhaps there is another way around this problem.
Justin
Javier de la Torre wrote:
Hi all,
I am starting to play with Geoserver and until now I have tosay
I
am impressed by how easy is to install and configure,
congratulations!!
But when trying to edit a schema.xml file to adapt the outputof
a
WFS service I get into problems. In the documentation it says:
"The last file is not always present, it is the schema.xmlfile.
This is a tricky file - if present it is used directly for a
DescribeFeatureType response, but configuration information isalso
read from it. If it is absent then the DescribeFeatureType
response
is automatically generated by GeoServer with the default
AttributeTypes of the DataStore (such as the columns of thetable)."
But if I edit manually this file Geoserver does not take care
of
it
for producing the GML of the features, but it does for the
DescribeFeatureType....
And the Graphical interface automatically overwrite to thedefault
one...
Do you know what is going on?
Thanks in advance.
Javier.
-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc. Get CertifiedToday
Register for a JBoss Training Course. Free Certification Exam
for All Training Attendees Through End of 2005. For more infovisit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc. Get CertifiedToday
Register for a JBoss Training Course. Free Certification Exam
for All Training Attendees Through End of 2005. For more infovisit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users----------------------------------------------------------
This mail sent through IMP: https://webmail.limegroup.com/----------------------------------------------------------
This mail sent through IMP: https://webmail.limegroup.com/