[Geoserver-devel] Geosync Community Module

Hi all,

I'd like to create a community module to house the code for Justin's and my work on geosynchronization. This is still mostly exploratory, but a general description of what we're trying to accomplish is on the wiki at http://geoserver.org/display/GEOS/GeoSync+and+Transaction+History+Feed .

The code I have still needs a bit of work to achieve basic functionality; I'm planning to add the module by tomorrow morning.

-David Winslow

+1 from me :).

David Winslow wrote:

Hi all,

I'd like to create a community module to house the code for Justin's and my work on geosynchronization. This is still mostly exploratory, but a general description of what we're trying to accomplish is on the wiki at http://geoserver.org/display/GEOS/GeoSync+and+Transaction+History+Feed .

The code I have still needs a bit of work to achieve basic functionality; I'm planning to add the module by tomorrow morning.

-David Winslow

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4007,47cd7ca156908992556831!

--
Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com

+1 from me as well, if you're looking for more votes.

--saul

On Tue, 2008-03-04 at 11:44 -0500, David Winslow wrote:

Hi all,

I'd like to create a community module to house the code for Justin's and
my work on geosynchronization. This is still mostly exploratory, but a
general description of what we're trying to accomplish is on the wiki at
http://geoserver.org/display/GEOS/GeoSync+and+Transaction+History+Feed .

The code I have still needs a bit of work to achieve basic
functionality; I'm planning to add the module by tomorrow morning.

-David Winslow

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

I think geosynchronisation is in fact the "killer app" for WFS,
althouygh not in the sense of RSS feeds but in terms of subscription
to data sets maintained by others.

This in turn means that we will care about the data, and that in terms
implies the necessity for community-schema support - otherwsie we have
a situation where the client must choose the identical technology and
schema as the data provider, and this is not tenable if you want to
get data from two or more sources.

So, IMHO taking geosycnh seriously means realising the
community-schema support on trunk first, and I'd like to see how the
geosynch module will work with ISO feature types. Otherwise we are
going to be in a position of endlessly hacking the one-namespace fits
all approach (which is a poor subset even of true "simple features" )
to cope with namespaces, attributes, column names mappings, case
sensistivity of data stroe issues etc.

i.e. +1 to do this properly, implementing against ISO features in
trunk, -1 to implement it against simple features.

Rob Atkinson

On Wed, Mar 5, 2008 at 3:44 AM, David Winslow <dwinslow@anonymised.com> wrote:

Hi all,

I'd like to create a community module to house the code for Justin's and
my work on geosynchronization. This is still mostly exploratory, but a
general description of what we're trying to accomplish is on the wiki at
http://geoserver.org/display/GEOS/GeoSync+and+Transaction+History+Feed .

The code I have still needs a bit of work to achieve basic
functionality; I'm planning to add the module by tomorrow morning.

-David Winslow

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Reviewing now:
- the atom feed of transactions is similar to what galdos does to cluster their WFS instances (note they were forwarding the entire transaction request around their cluster)
- you will need to publish the transaction response as well (so that FeatureID assignment can be kept in sync between nodes). That is we need both sides of the client / server conversation in order to have a successful solution.
- for your security concern; publishing a partial transaction request would break the use case of clients that want to update their own local cache; could use use security credentials when the feed is request and do per attribute screening? probably ... I would treat security as a seperate concerns that geoserver needs to manage at some point.
- for the history feed this sounds a little bit like the versioning postgis work; can you resuse any of the formalism defined there in terms of history requirements
- you need to consider locking as part of the grand plan; easiest approach is to add the LockRequest messages to the atom feed; note the other end is already taken care of as part of the Transaction request / response.

So +1 on the community module; and please feel free to ask me if you have questions about my comment above.
Jody

Hi all,

I'd like to create a community module to house the code for Justin's and my work on geosynchronization. This is still mostly exploratory, but a general description of what we're trying to accomplish is on the wiki at http://geoserver.org/display/GEOS/GeoSync+and+Transaction+History+Feed .

The code I have still needs a bit of work to achieve basic functionality; I'm planning to add the module by tomorrow morning.

-David Winslow

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
  

Rob Atkinson ha scritto:

I think geosynchronisation is in fact the "killer app" for WFS,
althouygh not in the sense of RSS feeds but in terms of subscription
to data sets maintained by others.

This in turn means that we will care about the data, and that in terms
implies the necessity for community-schema support - otherwsie we have
a situation where the client must choose the identical technology and
schema as the data provider, and this is not tenable if you want to
get data from two or more sources.

So, IMHO taking geosycnh seriously means realising the
community-schema support on trunk first, and I'd like to see how the
geosynch module will work with ISO feature types. Otherwise we are
going to be in a position of endlessly hacking the one-namespace fits
all approach (which is a poor subset even of true "simple features" )
to cope with namespaces, attributes, column names mappings, case
sensistivity of data stroe issues etc.

i.e. +1 to do this properly, implementing against ISO features in
trunk, -1 to implement it against simple features.

Rob, your opinion is appreciated but that -1 seem too strong of a
judgement.
Community schema and geosynch certainly play well together, cs
makes geosynch more widely applicable for sure, yet, from a
technological standpoint they are completely orthogonal, you
can implement one without having the other.

Community schema should stand on its own good, it makes a lot
of sense by itself, it needs a strong implementation, one
good enough to be merged into gt2 trunk and gs trunk (i.e.
solid and not too disruptive).

You're a PSC member, voting -1 to a proposal makes a lot harder
to have it implemented. You cannot force other people to implement
community schema in order to have another functionality there in
general, especially when there is no technical reason to do so.
Geosynch would certainly be a lot weaker without community schema,
agreed, but from there to say it's useless well, there's a long
stretch.

Cheers
Andrea

Actually its +1 to suing the ISOFeature model. -1 to creating even
more code that needs porting later. There shouldnt need to be a
dependency on community schema support as far as I can see. Community
schema support should just provide for rich data stores, and the
encoder should handle the outputs.

Persuade me otherwise - i.e. that we should create more porting work
by building new modules against to-be-deprecated APIs- and I'll
happily change it.

Rob

On Wed, Mar 5, 2008 at 9:39 PM, Andrea Aime <aaime@anonymised.com> wrote:

Rob Atkinson ha scritto:

> I think geosynchronisation is in fact the "killer app" for WFS,
> althouygh not in the sense of RSS feeds but in terms of subscription
> to data sets maintained by others.
>
> This in turn means that we will care about the data, and that in terms
> implies the necessity for community-schema support - otherwsie we have
> a situation where the client must choose the identical technology and
> schema as the data provider, and this is not tenable if you want to
> get data from two or more sources.
>
> So, IMHO taking geosycnh seriously means realising the
> community-schema support on trunk first, and I'd like to see how the
> geosynch module will work with ISO feature types. Otherwise we are
> going to be in a position of endlessly hacking the one-namespace fits
> all approach (which is a poor subset even of true "simple features" )
> to cope with namespaces, attributes, column names mappings, case
> sensistivity of data stroe issues etc.
>
> i.e. +1 to do this properly, implementing against ISO features in
> trunk, -1 to implement it against simple features.

Rob, your opinion is appreciated but that -1 seem too strong of a
judgement.
Community schema and geosynch certainly play well together, cs
makes geosynch more widely applicable for sure, yet, from a
technological standpoint they are completely orthogonal, you
can implement one without having the other.

Community schema should stand on its own good, it makes a lot
of sense by itself, it needs a strong implementation, one
good enough to be merged into gt2 trunk and gs trunk (i.e.
solid and not too disruptive).

You're a PSC member, voting -1 to a proposal makes a lot harder
to have it implemented. You cannot force other people to implement
community schema in order to have another functionality there in
general, especially when there is no technical reason to do so.
Geosynch would certainly be a lot weaker without community schema,
agreed, but from there to say it's useless well, there's a long
stretch.

Cheers
Andrea

Rob Atkinson ha scritto:

Actually its +1 to suing the ISOFeature model. -1 to creating even
more code that needs porting later. There shouldnt need to be a
dependency on community schema support as far as I can see. Community
schema support should just provide for rich data stores, and the
encoder should handle the outputs.

Persuade me otherwise - i.e. that we should create more porting work
by building new modules against to-be-deprecated APIs- and I'll
happily change it.

Well, while asking to use Feature instead of SimpleFeature seems a reasonable request, its feasibility really depends on how the
prototype is implemented.
I mean, if the prototype stores the feed in a well known feature type
in GeoServer, you're using the DataStore API, so you're stuck to
use SimpleFeature. The atom encoding is in fact already in the GS
code base since 1.5.x, thought we did not make it public because
it's not as well tested as RSS, and it uses SimpleFeature as well
(see the 1.5.x branch to have proof of that, http://svn.codehaus.org/geoserver/branches/1.5.x/geoserver/wms/src/main/java/org/vfny/geoserver/wms/responses/map/georss/)

So this leaves me wondering what part of the geosync module
could be actually implemented using ISO Feature.

Cheers
Andrea

A couple of things to note:

The first cut of our geosync stuff for the cite testbed wont even work with features. Its simply going to listen to transactions, serialize them out somehow, and then cough them up in an RSS feed on demand.

I agree with starting to write code against normal Feature when we can. I don't agree with halting all new work based on the argument that complex features is not on trunk yet.

Rob is of course entitled to vote -1, but remember that the process states that a -1 vote can only be factored into a proposal if its reasonable. And i am afraid that waiting for complex features to be implemented and stable on trunk is a big wait.

Also remember that the entire point of the community module stuff is to keep the entry barrier as low as possible for contributors. We can probably hold off the real debates until this stuff makes it way to the core.

My 2c.

-Justin

Andrea Aime wrote:

Rob Atkinson ha scritto:

Actually its +1 to suing the ISOFeature model. -1 to creating even
more code that needs porting later. There shouldnt need to be a
dependency on community schema support as far as I can see. Community
schema support should just provide for rich data stores, and the
encoder should handle the outputs.

Persuade me otherwise - i.e. that we should create more porting work
by building new modules against to-be-deprecated APIs- and I'll
happily change it.

Well, while asking to use Feature instead of SimpleFeature seems a reasonable request, its feasibility really depends on how the
prototype is implemented.
I mean, if the prototype stores the feed in a well known feature type
in GeoServer, you're using the DataStore API, so you're stuck to
use SimpleFeature. The atom encoding is in fact already in the GS
code base since 1.5.x, thought we did not make it public because
it's not as well tested as RSS, and it uses SimpleFeature as well
(see the 1.5.x branch to have proof of that, http://svn.codehaus.org/geoserver/branches/1.5.x/geoserver/wms/src/main/java/org/vfny/geoserver/wms/responses/map/georss/)

So this leaves me wondering what part of the geosync module
could be actually implemented using ISO Feature.

Cheers
Andrea

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4007,47ce8974166971096210785!

--
Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com

Yeah, that's my take too, when there's a gsip to make geosync a part of trunk then we can evaluate waiting on ISO feature model ect. This is just a proposal to get a community module, to experiment with some ideas, most of the code may be refactored anyways, to line up with versioning and the like, so it shouldn't be a big deal to refactor for full feature support. Also note that porting to the new feature model is not all that much work from the geoserver perspective. To wait months for the new feature model to stabilize when it'd just be a few hours to port over seems like a bit much to me.

best regards,

Chris

Justin Deoliveira wrote:

A couple of things to note:

The first cut of our geosync stuff for the cite testbed wont even work with features. Its simply going to listen to transactions, serialize them out somehow, and then cough them up in an RSS feed on demand.

I agree with starting to write code against normal Feature when we can. I don't agree with halting all new work based on the argument that complex features is not on trunk yet.

Rob is of course entitled to vote -1, but remember that the process states that a -1 vote can only be factored into a proposal if its reasonable. And i am afraid that waiting for complex features to be implemented and stable on trunk is a big wait.

Also remember that the entire point of the community module stuff is to keep the entry barrier as low as possible for contributors. We can probably hold off the real debates until this stuff makes it way to the core.

My 2c.

-Justin

Andrea Aime wrote:

Rob Atkinson ha scritto:

Actually its +1 to suing the ISOFeature model. -1 to creating even
more code that needs porting later. There shouldnt need to be a
dependency on community schema support as far as I can see. Community
schema support should just provide for rich data stores, and the
encoder should handle the outputs.

Persuade me otherwise - i.e. that we should create more porting work
by building new modules against to-be-deprecated APIs- and I'll
happily change it.

Well, while asking to use Feature instead of SimpleFeature seems a reasonable request, its feasibility really depends on how the
prototype is implemented.
I mean, if the prototype stores the feed in a well known feature type
in GeoServer, you're using the DataStore API, so you're stuck to
use SimpleFeature. The atom encoding is in fact already in the GS
code base since 1.5.x, thought we did not make it public because
it's not as well tested as RSS, and it uses SimpleFeature as well
(see the 1.5.x branch to have proof of that, http://svn.codehaus.org/geoserver/branches/1.5.x/geoserver/wms/src/main/java/org/vfny/geoserver/wms/responses/map/georss/)

So this leaves me wondering what part of the geosync module
could be actually implemented using ISO Feature.

Cheers
Andrea

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Andrea Aime wrote:

Well, while asking to use Feature instead of SimpleFeature seems a reasonable request, its feasibility really depends on how the
prototype is implemented.
  

You should be able to make use of Feature now; Gabriel did commit the DataAccess work. There is no reason
to wait for community schema to cough up an implementation; the existing implementations that produce SimpleFeature
will work fine...

You only need to use DataStore / SimpleFeature if you want to make additional assumptions about your data; Rob is asking
that you not do this.

I view my observation that Locking and the TransactionResponse needs to be considered as more serious complaints (ie the
idea as proposed has gaping holes) - but that was not a reason for me to vote -1 (in fact I voted +1 and asked that these
issues be looked at).

Cheers,
Jody

OK - now the issues are in the open, and have been addressed, I'm
happy to go with the flow. and give +1

Perhaps a little more visibility on where trunk is up to would be a
help - there is some point where the lowest entry bar stops being
working with an old API and porting later, or developing against the
new. IMHO this is not yet very transparent, and this raises the bar
in itself.

Rob

On Thu, Mar 6, 2008 at 1:45 AM, Chris Holmes <cholmes@anonymised.com> wrote:

Yeah, that's my take too, when there's a gsip to make geosync a part of
trunk then we can evaluate waiting on ISO feature model ect. This is
just a proposal to get a community module, to experiment with some
ideas, most of the code may be refactored anyways, to line up with
versioning and the like, so it shouldn't be a big deal to refactor for
full feature support. Also note that porting to the new feature model
is not all that much work from the geoserver perspective. To wait
months for the new feature model to stabilize when it'd just be a few
hours to port over seems like a bit much to me.

best regards,

Chris

Justin Deoliveira wrote:
> A couple of things to note:
>
> The first cut of our geosync stuff for the cite testbed wont even work
> with features. Its simply going to listen to transactions, serialize
> them out somehow, and then cough them up in an RSS feed on demand.
>
> I agree with starting to write code against normal Feature when we can.
> I don't agree with halting all new work based on the argument that
> complex features is not on trunk yet.
>
> Rob is of course entitled to vote -1, but remember that the process
> states that a -1 vote can only be factored into a proposal if its
> reasonable. And i am afraid that waiting for complex features to be
> implemented and stable on trunk is a big wait.
>
> Also remember that the entire point of the community module stuff is to
> keep the entry barrier as low as possible for contributors. We can
> probably hold off the real debates until this stuff makes it way to the
> core.
>
> My 2c.
>
> -Justin
>
> Andrea Aime wrote:
>> Rob Atkinson ha scritto:
>>> Actually its +1 to suing the ISOFeature model. -1 to creating even
>>> more code that needs porting later. There shouldnt need to be a
>>> dependency on community schema support as far as I can see. Community
>>> schema support should just provide for rich data stores, and the
>>> encoder should handle the outputs.
>>>
>>> Persuade me otherwise - i.e. that we should create more porting work
>>> by building new modules against to-be-deprecated APIs- and I'll
>>> happily change it.
>> Well, while asking to use Feature instead of SimpleFeature seems a
>> reasonable request, its feasibility really depends on how the
>> prototype is implemented.
>> I mean, if the prototype stores the feed in a well known feature type
>> in GeoServer, you're using the DataStore API, so you're stuck to
>> use SimpleFeature. The atom encoding is in fact already in the GS
>> code base since 1.5.x, thought we did not make it public because
>> it's not as well tested as RSS, and it uses SimpleFeature as well
>> (see the 1.5.x branch to have proof of that,
>> http://svn.codehaus.org/geoserver/branches/1.5.x/geoserver/wms/src/main/java/org/vfny/geoserver/wms/responses/map/georss/)
>>
>> So this leaves me wondering what part of the geosync module
>> could be actually implemented using ISO Feature.
>>
>> Cheers
>> Andrea
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2008.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>> _______________________________________________
>> Geoserver-devel mailing list
>> Geoserver-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel
>>
>>
>>
>
>