[Geoserver-devel] First draft of versioning WFS protocol extensions ready

Hi all,
today I've heavily updated the versioning WFS protocol
with some real proposals encoded in XML schema and explained
in text. You can find the results here:
http://docs.codehaus.org/display/GEOS/Versioning+WFS+-+Protocol+considerations

If you've already read the page, I suggest to re-read it from top to
bottom, all the WFS protocol part was heavily modified, lots of
details added and a few corrections in semantics too.

Let me know how do you like it, I'm going to implement the code
to handle that tomorrow (most will be generated from schema,
so changes are still easy).

Cheers
Andrea

Looks really great.

A few small feedbacks:

I like <FeatureCollection .... xsi:type="VersionedFeatureCollection" featureVersion="11502"> better than <VersionedFeatureCollection ...>. I feel like clients might barf if they get back a versionedFeatureCollection.

In GetDiffType you have a mismatch between documentation and xml for output format. Docs say:

The default value of 'application;xml; subtype=wfsv-transaction/1.1.0'

Schema says:

<xsd:attribute name="outputFormat" type="xsd:string" use="optional" default="text/xml; subtype=gml/3.1.1">

On RollbackType you might want to explain in the documentation why you're requiring a Filter. I imagine it's because you don't want people accidentally leaving one off and rolling back everything. But you should maybe explicitly state that, and tell them to do a 1=1 filter or some such if they really want to rollback everything.

But yeah, other than those everything looks really good. After you've implemented we should make a more friendly 'API' page, that is mostly examples instead of XSD snippets, as few people actually understand those. But this is a great explanation for OGC types.

best regards,

Chris

Andrea Aime wrote:

Hi all,
today I've heavily updated the versioning WFS protocol
with some real proposals encoded in XML schema and explained
in text. You can find the results here:
http://docs.codehaus.org/display/GEOS/Versioning+WFS+-+Protocol+considerations

If you've already read the page, I suggest to re-read it from top to
bottom, all the WFS protocol part was heavily modified, lots of
details added and a few corrections in semantics too.

Let me know how do you like it, I'm going to implement the code
to handle that tomorrow (most will be generated from schema,
so changes are still easy).

Cheers
Andrea

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:1003,45dc97a5120811365099012!

--
Chris Holmes
The Open Planning Project
http://topp.openplans.org

Chris Holmes ha scritto:

Looks really great.

A few small feedbacks:

I like <FeatureCollection .... xsi:type="VersionedFeatureCollection" featureVersion="11502"> better than <VersionedFeatureCollection ...>. I feel like clients might barf if they get back a versionedFeatureCollection.

Unfortunately that does not work very well with multiple versioning
data stores and non versioned feature types. The second proposal handles
that.
To keep things compatible with older clients, we can use a VersionedFeatureCollection only if the client showed knowledge of the
versioning api, that is, only if GetFeature did use featureVersion.

In GetDiffType you have a mismatch between documentation and xml for output format. Docs say:

The default value of 'application;xml; subtype=wfsv-transaction/1.1.0'

Schema says:

<xsd:attribute name="outputFormat" type="xsd:string" use="optional" default="text/xml; subtype=gml/3.1.1">

Ops. Will fix that.
Regarding diff, I'm having second thoughts on using Transaction as
the default output. While the structure is good, both delete and update have a full Filter element inside.
A server could theoretically log all wfs-v transaction requests and build an optimized and compact diff that leverages a lot filter spec, but I do believe this would raise the bar, both on the server side (a bit), and on the client side (too much to hope for a widespread adoption).

On RollbackType you might want to explain in the documentation why you're requiring a Filter. I imagine it's because you don't want people accidentally leaving one off and rolling back everything. But you should maybe explicitly state that, and tell them to do a 1=1 filter or some such if they really want to rollback everything.

Right. Explanations added, read that, because it's not really what you
were thinking about :slight_smile: That is, it wasn't meant for safety against big bad rollbacks, we do "unfiltered" rollbacks on subversion all the time, right?
There is still one important thing left out, that is, user identification: you may want to roll back stuff that a specific user did. Unfortunately user id is not in the feature type.
Now that I think about it, filtering against user is something you
may want to do in GetDiff, GetLog, and GetFeature too.
I can think of a couple of ways to handle that:
* have it in the filter anyways, as a phantom attribute, but say
   we do handle "user = xxx" only if it's a top level element of a
   "and" filter.
* extend GetFeature, GetLog, GetDiff and Transaction to have a
   userFilter element, with a list of user names in it. This does
   not allow, in general, for fancy filters that do restrict on users
   only when other conditions are met as well, but it's a lot simpler
   to implement. I do believe it's a big uglier, too.
* leave it all to the client, helping it a bit.
   Since user name is part of the GetLog output, and that the ChangeSets
   feature type is published anyways, have the client first
   filter against the user to find out revisions where the user did
   modify something, and then do a GetFeature against a range going from
   the lower to the higher.
   In order to make this work, we would need to revisit how version
   filtering works, and maybe go back to having a single featureVersion
   attribute that can be understood as:
   * empty, implying a single value, LAST
   * single value (value of features at that version)
   * range [v1,v2] -> all versions of features modified between
     v1 and v2 (bounds included?)
   * list {v1,v2,...,vn} -> all versions of features that has been
     modified at those specific versions
   * ALL (since the WFS spec reports it)
   The client would then issue a GetFeature asking for a list of
   revisions where the user has changes something, and then use that
   very list in a GetFeature, GetLog or GetDiff call.

Hum, the idea of using featureVersion being handled as a range or as a list is indeed interesting, it would reduce the amount of changes we
need in the XML structure even further (no DifferenceQuery element would be needed). I'm inclined to go down this road, unless I receive negative
feedback. The only thing I don't like very much, is that featureVersion
this way becomes complex enough to justify a sub-element for it,
instead of an attribute.
Especially since the single version can already be understood as
number, timestamp, branch:number, branch:timestamp (ugh..)

But yeah, other than those everything looks really good. After you've implemented we should make a more friendly 'API' page, that is mostly examples instead of XSD snippets, as few people actually understand those. But this is a great explanation for OGC types.

Will sure do. And provide a Geoserver with sample versioning requests too.

Cheers
Andrea

Andrea Aime ha scritto:

* leave it all to the client, helping it a bit.
   Since user name is part of the GetLog output, and that the ChangeSets
   feature type is published anyways, have the client first
   filter against the user to find out revisions where the user did
   modify something, and then do a GetFeature against a range going from
   the lower to the higher.
   In order to make this work, we would need to revisit how version
   filtering works, and maybe go back to having a single featureVersion
   attribute that can be understood as:
   * empty, implying a single value, LAST
   * single value (value of features at that version)
   * range [v1,v2] -> all versions of features modified between
     v1 and v2 (bounds included?)
   * list {v1,v2,...,vn} -> all versions of features that has been
     modified at those specific versions
   * ALL (since the WFS spec reports it)
   The client would then issue a GetFeature asking for a list of
   revisions where the user has changes something, and then use that
   very list in a GetFeature, GetLog or GetDiff call.

Hum, talked too soon on this one. The list does not make much sense
in diff and rollback operations, because we need a reference version
too. That's why we have fromVersion and toVersion, we need two
references to compute a diff.

I guess I'm leaning back to option 2, that is, an explicit
element for users. Well, let's see if the day bring counsel,
in the meantime I'll be playing with EMF and GTXML to setup parsing
of the extended wfs-v elements.

Cheers
Andrea

Andrea Aime wrote:

Chris Holmes ha scritto:

Looks really great.

A few small feedbacks:

I like <FeatureCollection .... xsi:type="VersionedFeatureCollection" featureVersion="11502"> better than <VersionedFeatureCollection ...>. I feel like clients might barf if they get back a versionedFeatureCollection.

Unfortunately that does not work very well with multiple versioning
data stores and non versioned feature types. The second proposal handles
that.
To keep things compatible with older clients, we can use a VersionedFeatureCollection only if the client showed knowledge of the
versioning api, that is, only if GetFeature did use featureVersion.

Ok, sounds good.

In GetDiffType you have a mismatch between documentation and xml for output format. Docs say:

The default value of 'application;xml; subtype=wfsv-transaction/1.1.0'

Schema says:

<xsd:attribute name="outputFormat" type="xsd:string" use="optional" default="text/xml; subtype=gml/3.1.1">

Ops. Will fix that.
Regarding diff, I'm having second thoughts on using Transaction as
the default output. While the structure is good, both delete and update have a full Filter element inside.
A server could theoretically log all wfs-v transaction requests and build an optimized and compact diff that leverages a lot filter spec, but I do believe this would raise the bar, both on the server side (a bit), and on the client side (too much to hope for a widespread adoption).

What is the alternative? I mean, in the alternative wouldn't it just be against featureids? So it shouldn't be a big problem to just have the filters always be featureid filters.

But I too was thinking that it might be better to have something more light weight as the default output. I suspect that usually with a diff you probably just want to see what the differences are. In svn the visual diffs are the same as what you'd submit, but I'd say WFS-T transaction requests may be a bit much for visual diffs. They should be one of the output formats, so that one could do one's own rollback if desired, but perhaps not the default. I imagine that it might be useful to have a few different output formats for this, json and html and text, for different clients to be able to just display the results.

On RollbackType you might want to explain in the documentation why you're requiring a Filter. I imagine it's because you don't want people accidentally leaving one off and rolling back everything. But you should maybe explicitly state that, and tell them to do a 1=1 filter or some such if they really want to rollback everything.

Right. Explanations added, read that, because it's not really what you
were thinking about :slight_smile:

Where's the explanation?

That is, it wasn't meant for safety against big bad rollbacks, we do "unfiltered" rollbacks on subversion all the time, right?
There is still one important thing left out, that is, user identification: you may want to roll back stuff that a specific user did. Unfortunately user id is not in the feature type.
Now that I think about it, filtering against user is something you
may want to do in GetDiff, GetLog, and GetFeature too.
I can think of a couple of ways to handle that:
* have it in the filter anyways, as a phantom attribute, but say
   we do handle "user = xxx" only if it's a top level element of a
   "and" filter.
* extend GetFeature, GetLog, GetDiff and Transaction to have a
   userFilter element, with a list of user names in it. This does
   not allow, in general, for fancy filters that do restrict on users
   only when other conditions are met as well, but it's a lot simpler
   to implement. I do believe it's a big uglier, too.
* leave it all to the client, helping it a bit.
   Since user name is part of the GetLog output, and that the ChangeSets
   feature type is published anyways, have the client first
   filter against the user to find out revisions where the user did
   modify something, and then do a GetFeature against a range going from
   the lower to the higher.
   In order to make this work, we would need to revisit how version
   filtering works, and maybe go back to having a single featureVersion
   attribute that can be understood as:
   * empty, implying a single value, LAST
   * single value (value of features at that version)
   * range [v1,v2] -> all versions of features modified between
     v1 and v2 (bounds included?)
   * list {v1,v2,...,vn} -> all versions of features that has been
     modified at those specific versions
   * ALL (since the WFS spec reports it)
   The client would then issue a GetFeature asking for a list of
   revisions where the user has changes something, and then use that
   very list in a GetFeature, GetLog or GetDiff call.

I think I like explicit user element as well.

Hum, the idea of using featureVersion being handled as a range or as a list is indeed interesting, it would reduce the amount of changes we
need in the XML structure even further (no DifferenceQuery element would be needed). I'm inclined to go down this road, unless I receive negative
feedback. The only thing I don't like very much, is that featureVersion
this way becomes complex enough to justify a sub-element for it,
instead of an attribute.
Especially since the single version can already be understood as
number, timestamp, branch:number, branch:timestamp (ugh..)

But yeah, other than those everything looks really good. After you've implemented we should make a more friendly 'API' page, that is mostly examples instead of XSD snippets, as few people actually understand those. But this is a great explanation for OGC types.

Will sure do. And provide a Geoserver with sample versioning requests too.

Sounds great.

Chris

Cheers
Andrea

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:1003,45dd613e252894820651628!

--
Chris Holmes
The Open Planning Project
http://topp.openplans.org

Chris Holmes ha scritto:

Andrea Aime wrote:

Chris Holmes ha scritto:

Looks really great.

A few small feedbacks:

I like <FeatureCollection .... xsi:type="VersionedFeatureCollection" featureVersion="11502"> better than <VersionedFeatureCollection ...>. I feel like clients might barf if they get back a versionedFeatureCollection.

Unfortunately that does not work very well with multiple versioning
data stores and non versioned feature types. The second proposal handles
that.
To keep things compatible with older clients, we can use a VersionedFeatureCollection only if the client showed knowledge of the
versioning api, that is, only if GetFeature did use featureVersion.

Ok, sounds good.

In GetDiffType you have a mismatch between documentation and xml for output format. Docs say:

The default value of 'application;xml; subtype=wfsv-transaction/1.1.0'

Schema says:

<xsd:attribute name="outputFormat" type="xsd:string" use="optional" default="text/xml; subtype=gml/3.1.1">

Ops. Will fix that.
Regarding diff, I'm having second thoughts on using Transaction as
the default output. While the structure is good, both delete and update have a full Filter element inside.
A server could theoretically log all wfs-v transaction requests and build an optimized and compact diff that leverages a lot filter spec, but I do believe this would raise the bar, both on the server side (a bit), and on the client side (too much to hope for a widespread adoption).

What is the alternative? I mean, in the alternative wouldn't it just be against featureids? So it shouldn't be a big problem to just have the filters always be featureid filters.

The point is, I cannot enforce these to be filters, thus clients
can really expect anything. For the prototype implementation, it does
not change things a log

But I too was thinking that it might be better to have something more light weight as the default output. I suspect that usually with a diff you probably just want to see what the differences are. In svn the visual diffs are the same as what you'd submit, but I'd say WFS-T transaction requests may be a bit much for visual diffs. They should be one of the output formats, so that one could do one's own rollback if desired, but perhaps not the default. I imagine that it might be useful to have a few different output formats for this, json and html and text, for different clients to be able to just display the results.

I can do text or html, but I guess these would work in a decent
way only for non geometric differences. Well, html could exploit
some fancy WMS getMap links to generate an image with the old and new feature, leveraging filters and some custom WMS extension that allows it to play with versioning (now, that would be real cool, and relatively
simple, add a FEATUREVERSION parameter, use FEATUREIDS filters, and show the two entities on two different layers with transparent fills).

On RollbackType you might want to explain in the documentation why you're requiring a Filter. I imagine it's because you don't want people accidentally leaving one off and rolling back everything. But you should maybe explicitly state that, and tell them to do a 1=1 filter or some such if they really want to rollback everything.

Right. Explanations added, read that, because it's not really what you
were thinking about :slight_smile:

Where's the explanation?

Grrr.... confluence ate my edits again... a lot of them. I hate it...
Going to rewrite things, it will take some time thought...

Cheers
Andrea

Chris Holmes ha scritto:

Where's the explanation?

Ok, added explanations, and the usage of "user" attribute in a few places (rollback, diff). Not sure this should be added to GetLog and
GetFeature too.

Another thing I've noticed, is that using DifferenceQuery implies having
attribute selection. Do we want to have partial diffs or partial
rollbacks only on selected attributes?

See also comments in italics here:
http://docs.codehaus.org/display/GEOS/Versioning+WFS+-+Protocol+considerations

Cheers
Andrea

Ops. Will fix that.
Regarding diff, I'm having second thoughts on using Transaction as
the default output. While the structure is good, both delete and update have a full Filter element inside.
A server could theoretically log all wfs-v transaction requests and build an optimized and compact diff that leverages a lot filter spec, but I do believe this would raise the bar, both on the server side (a bit), and on the client side (too much to hope for a widespread adoption).

What is the alternative? I mean, in the alternative wouldn't it just be against featureids? So it shouldn't be a big problem to just have the filters always be featureid filters.

The point is, I cannot enforce these to be filters, thus clients
can really expect anything. For the prototype implementation, it does
not change things a log

Ok.

But I too was thinking that it might be better to have something more light weight as the default output. I suspect that usually with a diff you probably just want to see what the differences are. In svn the visual diffs are the same as what you'd submit, but I'd say WFS-T transaction requests may be a bit much for visual diffs. They should be one of the output formats, so that one could do one's own rollback if desired, but perhaps not the default. I imagine that it might be useful to have a few different output formats for this, json and html and text, for different clients to be able to just display the results.

I can do text or html, but I guess these would work in a decent
way only for non geometric differences. Well, html could exploit
some fancy WMS getMap links to generate an image with the old and new feature, leveraging filters and some custom WMS extension that allows it to play with versioning (now, that would be real cool, and relatively
simple, add a FEATUREVERSION parameter, use FEATUREIDS filters, and show the two entities on two different layers with transparent fills).

That would be very cool, and could help adoption by making it even easier for clients to show visual diffs. And indeed in a much more efficient way, since they don't have to do any rendering or computation of diffs.

I think we'll just have to try out a few different ways to present diffs - this is definitely an area that will need research and feedback from clients. Do we have any idea how/if esri does such things in its versioning stuff?

Chris

Andrea Aime wrote:

Chris Holmes ha scritto:

Where's the explanation?

Ok, added explanations, and the usage of "user" attribute in a few places (rollback, diff). Not sure this should be added to GetLog and
GetFeature too.

Another thing I've noticed, is that using DifferenceQuery implies having
attribute selection. Do we want to have partial diffs or partial
rollbacks only on selected attributes?

I'd say that's overkill, at least for now. Looks like you've already updated the italics, sorry for the slow feedback. But all is looking good.

Chris

See also comments in italics here:
http://docs.codehaus.org/display/GEOS/Versioning+WFS+-+Protocol+considerations

Cheers
Andrea

!DSPAM:1003,45dde7c636211194215290!

--
Chris Holmes
The Open Planning Project
http://topp.openplans.org

Chris Holmes ha scritto:
...

I can do text or html, but I guess these would work in a decent
way only for non geometric differences. Well, html could exploit
some fancy WMS getMap links to generate an image with the old and new feature, leveraging filters and some custom WMS extension that allows it to play with versioning (now, that would be real cool, and relatively
simple, add a FEATUREVERSION parameter, use FEATUREIDS filters, and show the two entities on two different layers with transparent fills).

That would be very cool, and could help adoption by making it even easier for clients to show visual diffs. And indeed in a much more efficient way, since they don't have to do any rendering or computation of diffs.

I think we'll just have to try out a few different ways to present diffs - this is definitely an area that will need research and feedback from clients. Do we have any idea how/if esri does such things in its versioning stuff?

Hmm... there is something here about conflict and reconciliation (which
do require diff analysis):
http://gis.esri.com/library/userconf/proc02/pap0365/p0365.htm
http://webhelp.esri.com/arcgisdesktop/9.2/index.cfm?TopicName=Reviewing_conflicts

Mostly what you can expect, either attributes/geometries side by side,
or geometries one on top of the other.

Cheers
Andrea

Chris Holmes ha scritto:

Andrea Aime wrote:

Chris Holmes ha scritto:

Where's the explanation?

Ok, added explanations, and the usage of "user" attribute in a few places (rollback, diff). Not sure this should be added to GetLog and
GetFeature too.

Another thing I've noticed, is that using DifferenceQuery implies having
attribute selection. Do we want to have partial diffs or partial
rollbacks only on selected attributes?

I'd say that's overkill, at least for now.

Yeah, my conclusion as well. I changed DifferenceQuery not to extend
Query and removed the list of properties. All in all, the xsd are
simpler now (since the new DifferenceQuery basically has all the parameters I need in most operations, without the unneeded fat).

Cheers
Andrea