···
Our work in this area is going to be starting now but not really sure it will be ready to go for June… although I forecast good progress will be made. Although WFS 2.0 is going to be the last thing we get to chronologically.
What I suggest we do is both work on a branch for now to avoid stepping on each other in the short term.
-Justin
On Thu, Mar 20, 2014 at 1:35 AM, Sampo Savolainen <sampo.savolainen@anonymised.com> wrote:
Hi,
Great to hear the WFS data store support is moving forward. It’s reassuring to know there are other clients pushing for improvements in this sector. However the timeframe of this work does worry me a bit. My client needs a working implementation by June. Looking at how much refactoring and retrofitting there is to do, it seems that getting all the puzzle pieces fitting together by then is not feasible. Or what do you think?
At the moment, the best route for my work seems to build only the stored query support on top of wfs-ng while keeping a keen eye on the parallel development Justin outlined earlier (the steps 1-4). Then merge the work once we’re all finished. The biggest danger in this approach is that we might end up with a WFS 2.0 implementation that only supports stored queries.
Just to be sure: Is it possible to run both wfs and wfs-ng data stores in GeoServer, or does the entire server have to go with either one?
Sampo
–
Justin Deoliveira
Vice President, Engineering | Boundless
jdeolive@anonymised.com
@j_deolive
On Thu, Mar 20, 2014 at 1:02 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:
Ok, so it looks wfs-ng is the way to go. I think we will probably have enough budget on this project to have Niels do the work that we need there and implement some general clean up in order to get the module to supported status. So in terms of things our client can fund we have:
-
Implementing transactional support for WFS 1.0 and WFS 1.1
-
Porting over tests from old module, in particular the ones to ensure we work nicely with MapServer and TinyOWS
-
Push to supported status
-
WFS 2.0 (with transaction) support
Depending on how much time 1-3 take we may have to drop (4) but I think we have a large enough chunk of hours to tackle all of it. But it sounds like Sampo might be able to help out with (4) anyways.
@Ben: Support for complex features would be on your dime 
So do folks feel that is an acceptable plan? If so how do people want to want to collaborate? Niels will start work immediately, not sure what everyone else’s timeline is.
–
Sampo Savolainen
R&D Director, Spatineo Oy
sampo.savolainen@anonymised.com
+358-407555649
Linnankoskenkatu 16 A 17, 00250 Helsinki, Finland
www.spatineo.com, twitter.com/#!/spatineo
www.linkedin.com/company/spatineo-inc
This message may contain privileged and/or confidential information. If you
have received this e-mail in error or are not the intended recipient, you
may not use, copy, disseminate, or distribute it; do not open any
attachments, delete it immediately from your system and notify the sender
promptly by e-mail that you have done so.
On Wed, Mar 19, 2014 at 2:15 PM, Gabriel Roldan <groldan@anonymised.com> wrote:
Hi Andrea,
thanks for the quick reply, comments inline.
–
Justin Deoliveira
Vice President, Engineering | Boundless
jdeolive@anonymised.com
@j_deolive
On Wed, Mar 19, 2014 at 5:05 PM, Andrea Aime <andrea.aime@anonymised.com…> wrote:
No you’re not wrong, my memory is. So, correction: the wfs 1.1 datastore uses custom hand written StAX code to parse GetFeature responses.
thanks!
There are 1.x strategies for CubeWerx, GeoServer_pre_2.0 (as in the geoserver version), Ionic, and MapServer. TinyOWS didn’t exist by then or I didn’t know about it.
No custom strategies for WFS 2.0 so far.
Cheers,
Gabriel
–
Gabriel Roldán
Software Developer | Boundless
groldan@anonymised.com
@gabrielroldan
On Wed, Mar 19, 2014 at 8:38 PM, Gabriel Roldan <groldan@anonymised.com> wrote:
Hey sorry for the late reply,
The motivations to start wfs-ng were several:
- wfs uses two different xml “technologies” for wfs 1.0 and wfs 1.1. The one used for 1.0 is legacy (gt-xml) and for 1.1 it uses the geotools gt-xsd framework.
Afaik 1.1 uses a custom hand written pull parser, but maybe it is just for parsing XML?
Also, I believe it was supposed to read semi-complex features (pass me the term, as in features a bit more complex than the flat ones we support normally)?
I have vague memories that you tried gt-xsd streaming parsing, but found it was not ready for production usage? Maybe I’m wrong
- wfs also has two different DataStore implementations, for 1.0 based on the old AbstractDataStore, and for 1.1 being a pure interface implementation.
- at the time of starting wfs-ng at least, it was difficult to create unit and integration tests without live servers, which is worse due to the fact that (all?) servers behave differently, and we have strategy objects to deal with each one’s oddities. Yet most tests needed to be online, and “reference” online servers tend to disappear or be unavailable.
So the idea behind wfs-ng was to have a wfs datastore implementation that could be more easily maintained and extended to newer protocol versions, with the following design goals:
- there shall be one single WFSDataStore for all wfs versions, based on the new ContentDataStore “framework” the jdbc and other ones are based, given it’s the new “AbstractDataStore” which is practically deprecated.
- All the xml I/O and parsing, and the differencies between versions, should be handled by the new gt-xsd framework, yet that should remain an implementation detail as far as the datastore is concerned
- To keep building on top of existing code and enhance it instead of reinvent the wheel, the HTTP communication should be based on AbstractOpenWebService, on which also the wfs and aps clients are built.
- Easy of testing is a must, minimizing or nullifying the need for actual online servers
So we ended up with:
ContentDataStore AbstractOpenWebService
^ ^
| |
| |
| |
WFSContentDataStore → WFSClient → WFSStrategy
/
/
/ ________
AbstractWFSStrategy → | gt-xsd |
| |________|
StrictWFS_1_x_Strategy ___|
StrictWFS_2_0_Strategy ___|
Nice ASCII art! (and nice design, too)
Where WFSContentDataStore talks to the WFSClient in “OWS” terms like “issueRequest()”, “issueTransaction()”, “getRemoteTypeNames()”, etc; WFSClient asks the WFSStrategy to parse and encode requests and responses, but takes care of all the http communication and translation between DataStore objects and WFS protocol specific terms.
As for how good the wfs XX support is, its incomplete by all means. I think I got as far as using it for read only WFS 1.0 and 1.1 queries. The major blocker to get good 2.0 support at the time was that the gt-xsd bindings for 2.0 were too lacking. This is probably due to the fact that there are separate bindings for geoserver and geotools, and the geoserver ones received more attention as they were really needed there, but since there was no 2.0 client in geotools they were left incomplete.
As for transaction support, all the bits are in mostly in there. The one difficulty I found I think was a difference on how JDBC stores manage transactions (as ContentDataStore was originally written to support the jdbc-ng stores and hence was biased) and what I needed to do to replicate. jdbc ones hold transaction state at the jdbc Connection level so at commit() they’re just good to go, but I needed to hold uncommitted state in the client on a per transaction basis, yet I found that difficult to achieve with the per-featureType “ContentState” ContentDataStore uses. But I was also close to a solution when the work stalled. The ways to look at are the WFSLocalTransactionState and WFSRemoteTransactionState classes.
And IIRC the problem I was facing was that ContentDataStore’s “transaction states” would give me per-feature state diffs, forcing me issue one transaction element per feature for updates and deletes instead of batched as the wfs protocol supports, and I was trying to “merge” them into less WFS transaction elements.
That is to say, if you wanted to implement WFS 2 transaction support, all you’d need to do is implement StrictWFS_2_0_Strategy.createTransactionRequest(TransactionRequest request), where TransactionRequest is a module’s own type, and return the org.eclipse.emf.ecore.EObject that would then be encoded to xml by WFSRequest, which is part of the AbstractOpenWebService framework. As said, almost all in there already, but some 2.0 emf bits and lots of tests.
One question, in terms of strategies, are the various MapServer/TinyOWS/Cubewerx/addYourOwnVariantHere implemented in wfs-ng?
Cheers
Andrea
–
==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it