[Geoserver-devel] Proposal to implement limited virtual services

GeoServer GeoServer Developer
1

Hi all,

Recently OpenGeo has received funding to implement the idea of virtual services in a limited form. The basic idea is to provide service endpoints for each workspace so you can make requests like:

http://…/geoserver/topp/wfs?request=…

Basically what we currently do with workspace filters as a query parameter with some additions. The full GSIP is written up here:

http://geoserver.org/display/GEOS/GSIP+44+-+Virtual+services+with+workspaces

One thing to note is how this plays with resource publishing split since there is some overlap in this functionality and what resource pub split will provide. I wrote up some notes at the end of the proposal on this. But the gist of it is that the upgrade path should be quite smooth.

Actually this work fills one of the gaps that was shot in the resource pub proposal by Jody in that it did not explicitly specify the ability to specify a map in a url path as opposed to specifying it in the query string.

Comments and feedback welcome.

-Justin

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

2

I notice you talk about workspaces in one sentence then "a map" in
another sentence. My understanding is that workspaces are bound to
default namespaces (app-schema will get you out of that, but I've yet
to get my head around whether workspaces just become an impedence
overhead, or whether they effectively get bound to differend back-end
operational systems).

A "map" has a completely different set of semantics.

I see no particular reason a namespace or back-end operational system
is necessarily the only way of splitting up the geoserver services -
you are building a very inflexible solution IMHO. I would like to have
virtual services where you can put any resource the service should
offer, not just those defined by a namespace (either a made-up one or
an app-schema).

Can we not have a first-class concept of a service, and then map one
or more workspaces to it as a convenience mechanism?

I'd also like a single workspace to be available via multiple virtual
services to different audiences (simple, complete, and transactional
for example).

i.e. the mix of functionality for a single workspace is as valid a
reason for virtualisation as partitioning the set of resources across
workspaces.

Rob

On Tue, Dec 8, 2009 at 4:48 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi all,

Recently OpenGeo has received funding to implement the idea of virtual
services in a limited form. The basic idea is to provide service
endpoints for each workspace so you can make requests like:

http://…/geoserver/topp/wfs?request=…

Basically what we currently do with workspace filters as a query
parameter with some additions. The full GSIP is written up here:

http://geoserver.org/display/GEOS/GSIP+44+-+Virtual+services+with+workspaces

One thing to note is how this plays with resource publishing split since
there is some overlap in this functionality and what resource pub split
will provide. I wrote up some notes at the end of the proposal on this.
But the gist of it is that the upgrade path should be quite smooth.

Actually this work fills one of the gaps that was shot in the resource
pub proposal by Jody in that it did not explicitly specify the ability
to specify a map in a url path as opposed to specifying it in the query
string.

Comments and feedback welcome.

-Justin

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

3

Hi Justin:

Recently OpenGeo has received funding to implement the idea of virtual
services in a limited form. The basic idea is to provide service
endpoints for each workspace so you can make requests like:

http://…/geoserver/topp/wfs?request=…

So "topp" is the name of the service being offered in the above request. One question is the following valid.

http://…/geoserver/topp?Request=GetCapabilities&Service=WFS&Version=1.1…

I am trying to see if the service really is "topp" and the service supports one or more protocols...

Basically what we currently do with workspace filters as a query
parameter with some additions. The full GSIP is written up here:

http://geoserver.org/display/GEOS/GSIP+44+-+Virtual+services+with+workspaces

One thing to note is how this plays with resource publishing split since
there is some overlap in this functionality and what resource pub split
will provide. I wrote up some notes at the end of the proposal on this.
But the gist of it is that the upgrade path should be quite smooth.

Thanks Justin it is a well written proposal; the only section I had trouble with was the security implications. It is unclear if the approach you are advocating here is a change to the default security subsystem; or just an approach to use how to handle security at a future point in time? If it is a change for right now could you provide an example snippet of how to configure what you are describing?

Actually this work fills one of the gaps that was shot in the resource
pub proposal by Jody in that it did not explicitly specify the ability
to specify a map in a url path as opposed to specifying it in the query
string.

Thanks I picked up - and think the URL will be much more stable in this respect.

Comments and feedback welcome.

One assumption and a question

I assume this work needs to be made available in the stable 2.0.x series.
WIth that in mind what is your deadline for this work, and does the deadline include the functionality being issued as part of a release.

4

Jody Garnett wrote:

Hi Justin:

Recently OpenGeo has received funding to implement the idea of virtual services in a limited form. The basic idea is to provide service endpoints for each workspace so you can make requests like:

http://…/geoserver/topp/wfs?request=…

So "topp" is the name of the service being offered in the above request. One question is the following valid.

http://…/geoserver/topp?Request=GetCapabilities&Service=WFS&Version=1.1…

I am trying to see if the service really is "topp" and the service supports one or more protocols...

Yes, that is the idea. Of course at the moment we don't have the ability to turn off any of the services (aside from security).

Basically what we currently do with workspace filters as a query parameter with some additions. The full GSIP is written up here:

http://geoserver.org/display/GEOS/GSIP+44+-+Virtual+services+with+workspaces

One thing to note is how this plays with resource publishing split since there is some overlap in this functionality and what resource pub split will provide. I wrote up some notes at the end of the proposal on this. But the gist of it is that the upgrade path should be quite smooth.

Thanks Justin it is a well written proposal; the only section I had trouble with was the security implications. It is unclear if the approach you are advocating here is a change to the default security subsystem; or just an approach to use how to handle security at a future point in time? If it is a change for right now could you provide an example snippet of how to configure what you are describing?

The security implications are that if you plan to use a specific workspace endpoint, do not expect to be able to access layers from other workspaces. There will be zero configuration required.

Actually this work fills one of the gaps that was shot in the resource pub proposal by Jody in that it did not explicitly specify the ability to specify a map in a url path as opposed to specifying it in the query string.

Thanks I picked up - and think the URL will be much more stable in this respect.

Agreed, looks much nicer.

Comments and feedback welcome.

One assumption and a question

I assume this work needs to be made available in the stable 2.0.x series.
WIth that in mind what is your deadline for this work, and does the deadline include the functionality being issued as part of a release.

Yes, I will need this work to take place on the stable branch. As for when to release the date can be flexible. Since 2.0.1 is coming soon (or so i hear ;)) I would think targeting 2.0.2 would make most sense.

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

5

Rob Atkinson wrote:

I notice you talk about workspaces in one sentence then "a map" in
another sentence. My understanding is that workspaces are bound to
default namespaces (app-schema will get you out of that, but I've yet
to get my head around whether workspaces just become an impedence
overhead, or whether they effectively get bound to differend back-end
operational systems).

Yes, at this point in time a workspace == namespace, and we don't really have the notion of a map. When resource publishing split comes workspace will not be bound to a namespace. A namspace will simply be an attribute of a layer and not a container for layers. They will exist and be manged independently.

A "map" has a completely different set of semantics.

I see no particular reason a namespace or back-end operational system
is necessarily the only way of splitting up the geoserver services -
you are building a very inflexible solution IMHO. I would like to have
virtual services where you can put any resource the service should
offer, not just those defined by a namespace (either a made-up one or
an app-schema).

Yeah, so do I. And while I am at it I would also like to win the lottery tomorrow. We are at a middle ground. What you are asking for is coming when the full blown resource publishing split occurs. But this is an intermediate step.

Can we not have a first-class concept of a service, and then map one
or more workspaces to it as a convenience mechanism?

I'd also like a single workspace to be available via multiple virtual
services to different audiences (simple, complete, and transactional
for example).

i.e. the mix of functionality for a single workspace is as valid a
reason for virtualisation as partitioning the set of resources across
workspaces.

Rob

On Tue, Dec 8, 2009 at 4:48 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi all,

Recently OpenGeo has received funding to implement the idea of virtual
services in a limited form. The basic idea is to provide service
endpoints for each workspace so you can make requests like:

http://…/geoserver/topp/wfs?request=…

Basically what we currently do with workspace filters as a query
parameter with some additions. The full GSIP is written up here:

http://geoserver.org/display/GEOS/GSIP+44+-+Virtual+services+with+workspaces

One thing to note is how this plays with resource publishing split since
there is some overlap in this functionality and what resource pub split
will provide. I wrote up some notes at the end of the proposal on this.
But the gist of it is that the upgrade path should be quite smooth.

Actually this work fills one of the gaps that was shot in the resource
pub proposal by Jody in that it did not explicitly specify the ability
to specify a map in a url path as opposed to specifying it in the query
string.

Comments and feedback welcome.

-Justin

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

6

cool - just wanted to put the idea forward early to make sure its
considered. Perhaps a note to that effect in the GSIP?

On Wed, Dec 9, 2009 at 12:35 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Rob Atkinson wrote:

I notice you talk about workspaces in one sentence then "a map" in
another sentence. My understanding is that workspaces are bound to
default namespaces (app-schema will get you out of that, but I've yet
to get my head around whether workspaces just become an impedence
overhead, or whether they effectively get bound to differend back-end
operational systems).

Yes, at this point in time a workspace == namespace, and we don't really
have the notion of a map. When resource publishing split comes workspace
will not be bound to a namespace. A namspace will simply be an attribute of
a layer and not a container for layers. They will exist and be manged
independently.

A "map" has a completely different set of semantics.

I see no particular reason a namespace or back-end operational system
is necessarily the only way of splitting up the geoserver services -
you are building a very inflexible solution IMHO. I would like to have
virtual services where you can put any resource the service should
offer, not just those defined by a namespace (either a made-up one or
an app-schema).

Yeah, so do I. And while I am at it I would also like to win the lottery
tomorrow. We are at a middle ground. What you are asking for is coming when
the full blown resource publishing split occurs. But this is an intermediate
step.

Can we not have a first-class concept of a service, and then map one
or more workspaces to it as a convenience mechanism?

I'd also like a single workspace to be available via multiple virtual
services to different audiences (simple, complete, and transactional
for example).

i.e. the mix of functionality for a single workspace is as valid a
reason for virtualisation as partitioning the set of resources across
workspaces.

Rob

On Tue, Dec 8, 2009 at 4:48 AM, Justin Deoliveira <jdeolive@anonymised.com>
wrote:

Hi all,

Recently OpenGeo has received funding to implement the idea of virtual
services in a limited form. The basic idea is to provide service
endpoints for each workspace so you can make requests like:

http://…/geoserver/topp/wfs?request=…

Basically what we currently do with workspace filters as a query
parameter with some additions. The full GSIP is written up here:

http://geoserver.org/display/GEOS/GSIP+44+-+Virtual+services+with+workspaces

One thing to note is how this plays with resource publishing split since
there is some overlap in this functionality and what resource pub split
will provide. I wrote up some notes at the end of the proposal on this.
But the gist of it is that the upgrade path should be quite smooth.

Actually this work fills one of the gaps that was shot in the resource
pub proposal by Jody in that it did not explicitly specify the ability
to specify a map in a url path as opposed to specifying it in the query
string.

Comments and feedback welcome.

-Justin

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

7

On 08/12/09 21:35, Justin Deoliveira wrote:

Yes, at this point in time a workspace == namespace, and we don't really
have the notion of a map. When resource publishing split comes workspace
will not be bound to a namespace. A namspace will simply be an attribute
of a layer and not a container for layers. They will exist and be manged
independently.

[The crowd cheers!]

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

8

I am very glad to see this feature moving !

For what it's worth and I may not be aware of all the intricacies, but I have developed a poor man's solution for this. The concept is based on assigning Features to (internet) domains/URLs. The current implementation is done with a Servlet filter and XSLT. Configuration is done with a properties file. So basically the pool of Features (layers/resources) is managed within GS as usual, but from this pool sets of Features are available only on particular URLs. Sets may overlap. Like said, the implementation is a kludge (that is why I was hesitant to publish this as a Community Extension), e.g. filtering a GetCapabilities, rendering only the assigned Features, but the general idea is quite flexible. The split is not tied to work/namespaces. Maybe this is the same idea as suggested in other reactions.

A config file looks like this (* is all features):
site1.gis.nl=feature1,feature2,feature3
site2.gis.nl=feature2,feature5
www.gis.nl=*

As said this now works with domains (and XML formats only!) but could be URLs as well. From the outside these domains look like different GS instances. I can make all relevant files available if needed offcourse.

best,

Just van den Broecke

Ben Caradoc-Davies wrote:

On 08/12/09 21:35, Justin Deoliveira wrote:

Yes, at this point in time a workspace == namespace, and we don't really
have the notion of a map. When resource publishing split comes workspace
will not be bound to a namespace. A namspace will simply be an attribute
of a layer and not a container for layers. They will exist and be manged
independently.

[The crowd cheers!]

9

Yep - thats pretty much exactly the business requirement I was
prediciting - and you've even been motivated enought to build a layer
to enforce it!

Rob

On Thu, Dec 10, 2009 at 2:21 AM, Just van den Broecke
<just@anonymised.com> wrote:

I am very glad to see this feature moving !

For what it's worth and I may not be aware of all the intricacies, but I
have developed a poor man's solution for this. The concept is based on
assigning Features to (internet) domains/URLs. The current
implementation is done with a Servlet filter and XSLT. Configuration is
done with a properties file. So basically the pool of Features
(layers/resources) is managed within GS as usual, but from this pool
sets of Features are available only on particular URLs. Sets may
overlap. Like said, the implementation is a kludge (that is why I was
hesitant to publish this as a Community Extension), e.g. filtering a
GetCapabilities, rendering only the assigned Features, but the general
idea is quite flexible. The split is not tied to work/namespaces. Maybe
this is the same idea as suggested in other reactions.

A config file looks like this (* is all features):
site1.gis.nl=feature1,feature2,feature3
site2.gis.nl=feature2,feature5
www.gis.nl=*

As said this now works with domains (and XML formats only!) but could be
URLs as well. From the outside these domains look like different GS
instances. I can make all relevant files available if needed offcourse.

best,

Just van den Broecke

Ben Caradoc-Davies wrote:

On 08/12/09 21:35, Justin Deoliveira wrote:

Yes, at this point in time a workspace == namespace, and we don't really
have the notion of a map. When resource publishing split comes workspace
will not be bound to a namespace. A namspace will simply be an attribute
of a layer and not a container for layers. They will exist and be manged
independently.

[The crowd cheers!]

------------------------------------------------------------------------------
Return on Information:
Google Enterprise Search pays you back
Get the facts.
http://p.sf.net/sfu/google-dev2dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel