On Wed, Jan 25, 2012 at 10:21 AM, Andrea Aime
<andrea.aime@anonymised.com> wrote:
On Tue, Jan 24, 2012 at 9:56 PM, Gabriel Roldan <groldan@anonymised.com> wrote:
Hi all,
I'll be working on the wfs-ng module this and next weeks with the
following high level goals:
- port to ContentDataStore
- have a single WFSDataStore implementation with strategy objects for
the different versions, instead of one datastore per wfs version
- make it share the http client code with the wms and wps modules
- implement transaction support for wfs .1.1
- verify interoperability with a bunch of customer provided wfs instances
Nice. How are you going to deal with all the xml parsing/encoding required?
For any recognized oddity on each server a unit test will ensure the
sent request is like the server likes it. And for each server response
there'll be a mocked up response serving an xml file from the test
resources, plus any other needed thing like response headers etc to
account for server specific deviations from the spec.
As for xml tech, not enough resources to port all the WFS 1.0 xml
handling for the gt-xsd framework, so each protocol strategy object is
free to use whatever they want. For 1.1. we stick to gt-xsd for most
of the things. A possible exception being GetFeature response parsing.
Justin mentioned he has a refurbished streaming parser somewhere, so
it'd be worth giving it a try. Otherwise the StAX custom parser
already in the wfs 1.1 client would prevail.
Just ran into the question of what the earlier approach for "ng"
modules was before they're ready to replace the "original" ones, in
terms of connection parameter clashes.
Do we prefer "ng" modules:
1- to share the connection parameter names with the old ones, so that
they can be easily replaced
2- to temporarily use some different parameter name so that both
original and ng can be used at the same time, but fall back to the
original connection parameter when it's ready to replace the old
module
3- just use a different set of parameter names
I agree 2 is probably the best way
yup, thanks.
Additionally, it just occurred to me that the datastore could be
configured to use both a pooling http client (as the wms client can)
and to fetch features from the upstream server using multiple threads.
Default behavior would be to use a single thread. The old wfs 1.0
client forced spawning a new fetching thread per request, which didn't
scale, so we had to avoid that. I'm thinking a more modern approach
could be taken though, in order to have a fixed number of threads that
hit a given wfs server, and still get some performance improvement by
allowing a single request to use multiple threads to fetch contents,
as long as the upstream server supports paging. I'm not committed yet
to do that,but feedback would be much appreciated.
Sounds like a cool optimization, at the same time I'm wondering how much
impact it will actually have. Rationale:
- relying on paging it will work only against WFS 2.0 official
implementations,
or on WFS 1.0/1.1 vendor extensions. You did not mention WFS 2.0 above,
is that going to be implemented as well?
For vendor extensions, how do you recognize them? And how do you deal
with the issue of the first record being 0 or 1 depending on the
implementation
(GeoServer uses 0, MapServer uses 1, the spec is not clear)
- parallel requests will actually speedup things only if the bottleneck is
the CPU
and/or if each request is bandwidth capped, so that if you make parallel
requests you either get more CPU power or more total bandwidth
yeah, was mostly thinking out loud. Not something to do for this first
round, and certainly would need to evaluate the benefits and provide
enough configurability and perhaps heuristics adaptive to the actual
runtime conditions and/or enough profiling information as to be able
to tune on a case by case basis. Which would make that a project on
its own right, outside the current scope.
I agree it could be an interesting optimization for some cases, at the same
time making a truly interoperable client, that deals with WFS 1.1 servers
not flipping the axis as they should, usage of non EPSG codes (ESRI)
and so on, will be a "lot of fun" already 
Indeed... if you want to call it fun.
Cheers,
Gabriel
Cheers
Andrea
--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549
http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf
-------------------------------------------------------
--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.