R: [Geoserver-devel] Re: [Geotools-devel] Feature Versioning in/f or Geoserver (plus much much more)

This is quite near to what is my personal thinking.
See also this post of mine (sorry for the different Subject):
  "Ideas on versioning" (on the geotools-devel list)

Bye
Paolo Rizzi

-----Messaggio originale-----
Da: Chris Holmes [mailto:cholmes@anonymised.com]
Inviato: domenica 17 luglio 2005 17.53
A: dblasby@anonymised.com
Cc: geoserver-devel@lists.sourceforge.net;
geotools-devel@lists.sourceforge.net
Oggetto: [Geoserver-devel] Re: [Geotools-devel] Feature Versioning
in/for Geoserver (plus much much more)

Ok, first off, 'right on' - I think this can be incredibly
powerful, and
I like the thought of it hopefully getting more cross over with the C
projects. And I like that we're testing out pluggability,
hopefully to
incorporate the lessons learned here into geoserver 2.0

For this response I think I'm going to go with 'reply with an even
crazier idea', and yes, you can call me crazy too, but just
throwing it
out there to stir the juices - haven't thought it through all the way.

What about doing it under _and_ over, implement at the
datastore level,
but have geocollaborator expose it all using WMS and WFS datastores,
completely sitting in its own external project. But since it's all
done with datastores you could for example just plug-in a postgis
datastore instead of a WFS datastore, for example. It would
plug in to
more than just any OGC compliant service, it would also plug on top of
any (transactional) datastore supported by GeoTools.

The main place this is coming from is my fear of XSLT. This may be an
irrational fear, but I guess I worry that XSLT is in many ways another
whole programming language, and a very verbose and not easy to
understand one at that. Or perhaps I'm not clear exactly what will be
done in geotools and what will be done in xslt, where that
line will be
drawn.

As an alternative couldn't you just use the GeoServer code to read WFS
requests, put them into a Query object, and then write java code to do
the appropriate actions with that Query? Reformulate them into new
Queries to be sent to the base WFSes. The reformulation is done in
Java, which is more widely known than XSLT.

The upside to this is that if we're just making Queries, then they can
be directed at Datastores - GeoCollaborator code would just always use
a WFS datastore. The downside is that WFS datastore seems to be in a
state of transition with the new xdo code, and we're still
not sure how
maintainable it is. Of course the upside of that is it will force us
to do a lot more bug fixing on it, and end up with a really high
quality WFS datastore/parser.

We also could consider doing both - if we're going to encourage more
value added plug-ins it could actually be nice to give people the
option to do xslt or Java - some of the C types may be more
comfortable
with xslt (though some of the mapserver crowd seem to really not like
xml). Make like the logging plug-in available as both, for
contributors to use as an example of either xslt or pure java style.

I don't think slotting into GeoServer is a good idea, the validation
code should either move up or down the stack.

The over/under approach also has the advantage of being able to have
GeoCollaborator functionality stand on its own, but also to be easier
integrated as a GeoServer 2.0 plugin, so that admins could just do one
install.

Chris

Quoting dblasby@anonymised.com:

> I've quite interested in getting Geoserver doing
FeatureVersioning so
> that when we (TOPP) start doing
> Geo-Wiki/GeoCollaborator/Public-Participation GIS stuff that can
> track
> and rollback changes, and "view" the dataset at a particular
> point-in-time.
>
> Personally, I think this (in itself) will be extremely powerful.
>
> The design is a bit tricky - but I think the actual versioning code
> will
> not be too difficult.
>
>
>
> There are a few meta-design issues that need to be resolved first.
> I've
> indirectly talked to a few people about this, but I, unfortunately,
> haven't had time to put more than cursory thought into it.
>
> In the email, below, I argue for a system (probably built using
> geotools
> & XSLT) that can "extend" the functionality of already existing
> services. All these services could be directly added to
Geoserver or
> Geotools, but by building it in this system I feel its (1)
easier and
> (2) allows more cross-project collaborations.
>
> I feel the Java GIS stuff is totally separated from the non-Java GIS
> stuff, and this makes us "marginalized". Tyler recently summarized
> the
> major open-source mapping projects and didn't mention a single Java
> based one. Since the system I describe below can be used to give
> extra functionality to ALL systems that implements the OGC services
> (ie. mapserver and - shock - ARCIMS) I believe we'll have
some common
> ground to share.
>
> I'm hoping that we'll be able to entice people who want to continue
> using Mapserver (or whatever they're currently using) to know & use
> the
> java project, and also so that people using the non-java projects
> could
> start contributing to the java projects. Also, I feel,
we'll be able
> to have a "full" package of service built on top of any subset of
> implementations.
>
> I guess this is sounding more like the open-SDI/Geo-Web application
> framework, but I want to stress that the system I'm describing here
> is
> supposed to be quite simple - adding new services or complex
> processes
> would be better implemented in the 2.0 (easy-to-add-new-services)
> geoserver.
>
> I'm hoping to get a little feedback and see what people think about
> this
> type of thing. Feel free to call me crazy, or better yet,
reply with
> an even crazier idea.
>
>
> dave
>
> -----------------------------------------------------
>
> I've been thinking a little bit about versioning. There's 3 places
> you
> can put it:
>
> 1. in a datastore ("underneath geoserver")
> 2. inside geoserver (like the way the current validation slots into
> the
> Transaction java code)
>
> 3. on top of geoserver - this is a bit weird, but it makes good
> sense.
> Basically, have a component that sits outside geoserver
that takes in
> WMS/WFS requests and reforms its (i.e. XSLT) and sends them
back down
> to the actual WFS/WMS server. You can implement a large number of
> actual services using this method. The main advantage is you can
> make
> it fairly simple, and it can be used to extend any WFS (not just
> geoserver) into a value-added WFS.
>
> Here's some thoughts I wrote a while ago on it (GeoCollaborator =
> this
> "on top of geoserver" component):
>
> There's a core GeoCollaborator service. This is where most of the
> work will actually go. The idea behind the service is very simple -
> it takes in an OGC request, modifies it, and then spits it back out
> to
> other OGC servers that actually handle the request.
>
> That means that outside - either above it or below it - is only
> ever exposed to OGC requests and responses. This is key because you
> get to extend all the power of those servers and people are free to
> pick and choose what product (ie. geoserver, mapserver, degree, or a
> commercial product) that will actually handle the work. Plus, you
> can
> use any of the client applications (like udig and mapbuilder) that
> "speak" OGC. It provides lots of power without actually
complicating
> anything.
>
>
> Some of the services it could offer would be:
>
> "logging" (who's viewing what and where) - any incoming read
> request get transformed into a WFS write (storing the layer, user,
> time
> and location) AND the read request.
>
> "validation" - i.e. an insert request for a road is transformed
> into
> an
> "is this road in the ocean?" request before the actual insert is
> performed.
>
> "fixing" - ie. for roads, if two roads cross and are not
> topologically noded, then pre-node them before inserting them.
>
> "versioning" - for rollback, point-in-time, and the like.
>
> "extended functionality" -- for example, Gabriel wanted to add a
> "indirect image response" to the Geoserver WMS to make it easier to
> use
> SLD-POST in a browser. Basically, you send a SLD-POST
request to the
> WMS and it returns a little XML fragment that says "go HTTP-GET your
> image here". This is MUCH easier to actually use in a browser with
> <IMG> tags (which do not allow HTTP-POST).
>
> You would send an SLD-POST to the Geocollaborator and it will echo
> the
> request to the underlying WMS (or, if it doesn't support SLD-POST,
> convert it to an SLD-GET request) and capture the image
response (or,
> for dealing with dynamic data, remember how to make the WMS
request).
> It then sends back the small XML response saying how to HTTP-GET the
> image.
>
> This could then be used to "wrap" any WMS instead of just adding the
> functionality to Geoserver.
>
> "layer creation"
>
> "user creation & security"
>
> "virtual datastores" -- these can be implemented by transforming
> the
> results of a WFS request.
>
> ...and so on...
>
>
> All of these would be fairly simple and the main core will just be
> a simple XSLT transformer with a nice plugin architecture. Most
> people
> will use it "as is", the more technically daring could mix-and-match
> pre-made (from other projects) plugsin, hard core geo-hackers could
> make their own plugins
>
> The core is focused to do what is does really well. For more
> 'complex' things, you'd have to go to something like geoserver to
> actually make the service.
>
>
>
> I haven't really put much thought into this; whats your first
> reaction?
> Personally, I like the "under" or "on top of" position.
>
> The "under" has the advantage of probably being easiest,
and then all
> Geotools-based programs could use it.
>
> The "on top of" has the advantage of extending all OGC WFS/WMS
> services
> instead of just Geoserver/Geotools. It also makes a great place to
> put
> the "rollback" "look for changes" "get point-in-time view" UI. Its
> also a nice place to allow people to add simple plugins to extend
> functionality (although the "new" easy-to-plugin-to-geoserver might
> be
> a better place for complex things).
>
> dave
>
>
> ----------------------------------------------------------
> This mail sent through IMP: https://webmail.limegroup.com/
>

----------------------------------------------------------
This mail sent through IMP: https://webmail.limegroup.com/

-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

P. Rizzi Ag.Mobilità Ambiente wrote:

This is quite near to what is my personal thinking.
See also this post of mine (sorry for the different Subject):
  "Ideas on versioning" (on the geotools-devel list)

Since it has come up in a couple of peoples ideas. With respect to making a VesioningDataStore... Lets not.

Please lets just treat versioning as a first class concern and change the DataStore API. The OGC has version and timestamp modeled as being part of what defines a Feature (just the same as feature ID).

So lets make sure the geotools system is rich enough to capture this idea of a Feature, and lets change the DataStore API to take advantage of that fact.

Jody

Quoting Jody Garnett <jgarnett@anonymised.com>:

P. Rizzi Ag.Mobilità Ambiente wrote:
> This is quite near to what is my personal thinking.
> See also this post of mine (sorry for the different Subject):
> "Ideas on versioning" (on the geotools-devel list)
>
Since it has come up in a couple of peoples ideas. With respect to
making a VesioningDataStore... Lets not.

Please lets just treat versioning as a first class concern and change
the DataStore API. The OGC has version and timestamp modeled as
being
part of what defines a Feature (just the same as feature ID).

So lets make sure the geotools system is rich enough to capture this
idea of a Feature, and lets change the DataStore API to take
advantage
of that fact.

I'm less than convinced that the OGC models are complete enough to do
versioning - it really feels like the pushed it in as an after thought.
Indeed, WFS is what we use for our datastore model, and the WFS leaves
out versioning except for just talking about it in a GetFeature, no
mention in transactions or anything like that. That said, I'm not
against changing the datastore api to deal with this, as long as we go
typical datastore style, so that all datastores can handle it equally
well, and those that can leverage their backend optimizations do.

But it does get tricky - with rdbms we can presumely just create another
meta table to track versioning, but can we do that with shapefiles? Or
do we embed a db somewhere in there? Keeping it in separate wrapper
datastores can make it explicit what's going on -
DerbyVersioningDataStore, NativeJDBCDataStore, ect. Though I suppose
we could still have those backing a coherent api.

Chris

Jody

----------------------------------------------------------
This mail sent through IMP: https://webmail.limegroup.com/