[Geoserver-devel] GeoServer Resources

[Tim_Schaub2]

Hey-

I'd be interested to hear thoughts on an architecture proposal for GeoServer:
http://geoserver.org/display/GEOSDOC/GeoServer+Resources

This builds off of work that David Winslow has been doing and has written up here:
http://geoserver.org/display/GEOSDOC/RESTful+User+Management+API
http://geoserver.org/display/GEOSDOC/RESTful+Configuration+API

Though on the surface it may not appear that different, I've written it up as though GeoServer has a single web interface - not separating out an API from a user interface. There are also a few new ideas in there that I'm interested to hear feedback on.

1) I think it would be cool to think big picture style about how a data config API and user managment API fit in to the rest of GeoServer's web interface. Largely, this is about tucking things in the /api path - which suggests that there should be a different interface for non-programmed users. I like the idea that GeoServer exposes a number of resources through one interface on the web - and that the same interface is useful for humans and machines.

2) I think it makes sense to have one set of resources related to configuring data (workspaces) and one set related to publishing layers (maps). The name "workspaces" captures the meaning of a resource for the configuration of layer collections a bit better for me than "folders" - where that collection can be a database connection, a disk based folder, or a group of remote layers via an OGC service. Workspaces (places to configure data typically restricted to an administrator) are distinguished from maps (collections of published layers typically accessible by an anonymous user).

3) I know (I'm pretty sure) the idea of collecting layers together in "maps" would be a new feature for GeoServer. I think this idea is one that most users of ESRI products (ArcIMS or ArcGIS Server) and MapServer come to expect. In addition to having access to the resources in a map (layers, users, projections, styles) through a RESTful interface, GeoServer could provide OGC services based on maps. So http://host/geoserver/maps/political/layers would provide a list of layers in a political map and http://host/geoserver/maps/political?SERVICE=WMS&… would be a WMS endpoint.

4) I also don't know squat about auth/auth in GeoServer, but it seems to me like it would be pretty handy to expose a "groups" resource (and/or "roles") next to the users resource. Then every other resource (with the exception of individual features perhaps) could have a "permissions" resource that described how each group (or role) was limited by verb (get, post, put, delete) - with an option for inheritance from a parent resource. I may be naive, but that seems like a pretty nice way to control security.

5) We're currently doing some work on style configuration. I like the idea of having a collection of styles at /geoserver/styles and then assigning named styles to layers through the /geoserver/maps/<name>/layers/<name>/styles resource. The styles could have a short name for WMS (because those are unique in the /geoserver/styles collection) and could also be resolved by it's complete URL.

Thanks for any feedback.
Tim

[Andrea_Aime3]

Wow, lots of stuff. I'll try to answer each item in its own
mail since I have the impression this will generate quite a thread

Tim Schaub ha scritto:

Hey-

I'd be interested to hear thoughts on an architecture proposal for GeoServer:
http://geoserver.org/display/GEOSDOC/GeoServer+Resources

This builds off of work that David Winslow has been doing and has written up here:
http://geoserver.org/display/GEOSDOC/RESTful+User+Management+API
http://geoserver.org/display/GEOSDOC/RESTful+Configuration+API

Though on the surface it may not appear that different, I've written it up as though GeoServer has a single web interface - not separating out an API from a user interface. There are also a few new ideas in there that I'm interested to hear feedback on.

1) I think it would be cool to think big picture style about how a data config API and user managment API fit in to the rest of GeoServer's web interface. Largely, this is about tucking things in the /api path - which suggests that there should be a different interface for non-programmed users. I like the idea that GeoServer exposes a number of resources through one interface on the web - and that the same interface is useful for humans and machines.

I like the idea of a single consistent REST api that goes from configuration to actual data.
A user interface based on the same structure (that is, one that
is completely based in rest style, that is, simply return back HTML
as the default resource encoding) may be appealing, thought I'm not sure
how you would handle in REST style a few typical use cases:
- per feature type service related configurations.
   As an example of service specific knob, have a per feature type
   maxFeature setting that is WFS specific. Or the styles, that in fact
   are WMS specific.
   As we add services to GeoServer, the list of service specific knobs
   will increase. We could have a special subpage for each, like:
   {featureType}/services/{serviceName} but wouldn't it make things a
   little too verbose (navigation wise)
- custom per data store creation pages. The current approach uses
   a generic form that no one really likes, it would be better to have
   custom, specific configuration pages used to create a new datastore.
   But there would be aids to create a new datastore, they would not
   be resources per se. How do you handle that?

Btw, I noticed now that the proposal page does not have the concept
of a datastore, that is, one of a connection to a database, remote wfs
server, or a file (and the same goes for coverages).

If you're following the mapserver approach, you'd have to specify over
and over again the same connection params for each layers, which I'd
find quite a bit cumbersome.
The current approach is cumbersome in its own way too, since you have
to manually go and register layers one by one, it would be a lot better
if after registering the datastore you had a list of all the possible
feature types, and you cherry pick the ones you want to expose
via GeoServer. Resource wise, it would be nice to have the following:

/geoserver/workspaces/<wsName>/datastores/{dsName}/<ftName>

that is, a way to expose feature types that are there, but aren't
still made available thru GeoServer. We could use a request to
turn each ftName on and off, when you turn it on, it becomes an
available layer in:

/geoserver/workspaces/<name>/layers

Same would happen for coverages and coverage stores.
The thing is, we'd need to make datastores and coveragestores
paths visible only to admin user types, whilst the layers part
would be visible to everybody (as a REST style "capabilities" offering).

Cheers
Andrea

[Andrea_Aime3]

Tim Schaub ha scritto:
...

2) I think it makes sense to have one set of resources related to configuring data (workspaces) and one set related to publishing layers (maps). The name "workspaces" captures the meaning of a resource for the configuration of layer collections a bit better for me than "folders" - where that collection can be a database connection, a disk based folder, or a group of remote layers via an OGC service. Workspaces (places to configure data typically restricted to an administrator) are distinguished from maps (collections of published layers typically accessible by an anonymous user).

I'm having troubles matching worskpaces and maps to what we have today.

Workspace seems to recall a little the idea of namespace, and a little
the idea of a data directory.

Namespaces are today the only organisational structure we have in GeoServer, each datastore and coveragestore is placed under a certain
namespace, and will be soon used to drive per layer security restrictions.
Unfortunately namespace by itself is not strong enough to replace
the concept of a data directory, since we don't have a per namespace
service and server configuration.
This leads users to deploy geoserver multiple times, in order to have
specific service configurations, like, for example, bilinear interpolation for a group of wms coverage layers, but not for the others, or have WFS-T enabled in one case but not on the other.

If we had the possibility to state a per namespace service level
configuration as well, it would make it possible to treat namespaces
as the organisational structure you called workspaces.

Also mind that namespaces are needed for wfs anyways, so we cannot
eliminate them. What I'm proposing is to make them strong enough
to represent the concept of workspace, a container that has
certain layers and certain service and server configurations: think
wms interpolation, antialiasing, logging level, but also set of users,
security policies and the like.

The major beneficial side effect of this would be that we won't
need anymore to deploy multiple geoserver just to handle different
user groups (think an ASP that wants to offer GeoServer hosting to
multiple customers).

Which brings me to the map concept, that seems to be modelled
after the MapServer mapfile. At first I thought it was like a layer
group (a single name used in wms calls to signify a stack of layers with
pre-confired styles), but looking at the REST api, it's not... or is it?

I'm wondering if we really need this concept or not. The workspaces
already provide a way to organise layers, why do we need a separate
map concept? Styles and projections could be very well attached
to workspace layers (which is what is happening today).

The only reason I see that may drive people to use the map concept is
to be able to lock down what styles you may use for a certain layer,
and have different lock downs for different users.
But it seems to me this defeats the purpose of an SDI as a way to
let people freely look at the data, combine it in different ways.
It would be ineffective too unless we disallow people to provide
the &sld parameter.
Finally, it would require quite a bit of changes in WMS code as well since, as far as I understand it, each "map" would have to be represented by a separate WMS service no?

I'm not saying no to the idea, but I'm wondering "where's the meat" in the map concept.

Cheers
Andrea

[Andrea_Aime3]

Tim Schaub ha scritto:
...

4) I also don't know squat about auth/auth in GeoServer, but it seems to me like it would be pretty handy to expose a "groups" resource (and/or "roles") next to the users resource. Then every other resource (with the exception of individual features perhaps) could have a "permissions" resource that described how each group (or role) was limited by verb (get, post, put, delete) - with an option for inheritance from a parent resource. I may be naive, but that seems like a pretty nice way to control security.

At the moment GeoServer security is pretty lame, it's based only on
the service and method, that is, you can say only certain users (roles
in fact, RBAC security: http://en.wikipedia.org/wiki/RBAC) can access
a certain service method, like in:
WFS.Transaction=WFS_T_USER,ADMINISTRATOR
where you state that the WFS Transaction method can be called only
by users having either WFS_T_USER or ADMINISTRATOR role.

If things go well (sponsoring wise) we should be able to have namespace
and layer based security, that is, something like:
topp.*.r=TOPP_USER
topp.*.w=TOPP_USER
topp.states.w=STATES_WRITER
stating that every layer in the topp namespace can be accessed only by
users that have the TOPP_USER role, with an override for topp:states
saying that only users with users with STATES_WRITER role can modify
the states layer.

There is no plan to have per feature or per property access control
(think restricted areas, or data columns that only certain users can see) at the moment.

REST wise, this could be represented as:
/worskspaces/<wsname>/layers/<layer>/permissions/read
/worskspaces/<wsname>/layers/<layer>/permissions/write
/worskspaces/<wsname>/permissions/read
/worskspaces/<wsname>/permissions/write
(where each read/write is a list of roles)
and as
/worskspaces/<wsname>/users/<user>/roles

Cheers
Andrea

[Andrea_Aime3]

Tim Schaub ha scritto:

5) We're currently doing some work on style configuration. I like the idea of having a collection of styles at /geoserver/styles and then assigning named styles to layers through the /geoserver/maps/<name>/layers/<name>/styles resource. The styles could have a short name for WMS (because those are unique in the /geoserver/styles collection) and could also be resolved by it's complete URL.

This pretty much matches what is happening these days... and it has
some troubles which we may want to fix... or not

The trouble is that some styles are completely layer agnostic, like polygon, point, line, giant_polygon and so on, whilst some work only
if the layer has certain attributes, like population.

The drawback of making them a single lot is that you don't know in
advance if a style is made to work with a single layer, or for
every layer. Then again, the same style could be used for a group
of layer that do share the same attributes, so maybe it's better
to leave things as they are today.

Opinions?
Cheers
Andrea

[Justin_Deoliveira]

I like the idea of a single consistent REST api that goes from configuration to actual data.
A user interface based on the same structure (that is, one that
is completely based in rest style, that is, simply return back HTML
as the default resource encoding) may be appealing, thought I'm not sure
how you would handle in REST style a few typical use cases:
- per feature type service related configurations.
   As an example of service specific knob, have a per feature type
   maxFeature setting that is WFS specific. Or the styles, that in fact
   are WMS specific.
   As we add services to GeoServer, the list of service specific knobs
   will increase. We could have a special subpage for each, like:
   {featureType}/services/{serviceName} but wouldn't it make things a
   little too verbose (navigation wise)
- custom per data store creation pages. The current approach uses
   a generic form that no one really likes, it would be better to have
   custom, specific configuration pages used to create a new datastore.
   But there would be aids to create a new datastore, they would not
   be resources per se. How do you handle that?

Agreed, while simply making the resource structure available as an html form or something is nice... i think it might be a bit limiting. Although I dont think I understand Tim fully on this point.

Btw, I noticed now that the proposal page does not have the concept
of a datastore, that is, one of a connection to a database, remote wfs
server, or a file (and the same goes for coverages).

I might be wrong here but I think a "workspace" corresponds to what we think of as a "datastore" now.

If you're following the mapserver approach, you'd have to specify over
and over again the same connection params for each layers, which I'd
find quite a bit cumbersome.

Right, this is how it works in MapServer, but I think here this is is different. The layers are "configured" once (in terms of connection parameters) on the workspace side, and then you can freely add and remove those layers to a map on the map side. The wms capabilities document then feeds off of a particular map.

The current approach is cumbersome in its own way too, since you have
to manually go and register layers one by one, it would be a lot better
if after registering the datastore you had a list of all the possible
feature types, and you cherry pick the ones you want to expose
via GeoServer. Resource wise, it would be nice to have the following:

/geoserver/workspaces/<wsName>/datastores/{dsName}/<ftName>

that is, a way to expose feature types that are there, but aren't
still made available thru GeoServer. We could use a request to
turn each ftName on and off, when you turn it on, it becomes an
available layer in:

/geoserver/workspaces/<name>/layers

The way I understand it is that when you create a workspace, the layers in it are not "published". They don't become published until you add those layers to a particular map.

Aside: I don't like exposing the idea of a "datastore" through this ui. Its a very abstract term that users have a hard time understanding. Same goes for "feature type".

Same would happen for coverages and coverage stores.
The thing is, we'd need to make datastores and coveragestores
paths visible only to admin user types, whilst the layers part
would be visible to everybody (as a REST style "capabilities" offering).

Cheers
Andrea

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4007,47eb6115187221137850744!

--
Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com

[David_Winslow]

On Thursday 27 March 2008 04:55:27 Andrea Aime wrote:

Wow, lots of stuff. I'll try to answer each item in its own
mail since I have the impression this will generate quite a thread

Tim Schaub ha scritto:
> Hey-
>
> I'd be interested to hear thoughts on an architecture proposal for
> GeoServer:
> http://geoserver.org/display/GEOSDOC/GeoServer+Resources
>
> This builds off of work that David Winslow has been doing and has
> written up here:
> http://geoserver.org/display/GEOSDOC/RESTful+User+Management+API
> http://geoserver.org/display/GEOSDOC/RESTful+Configuration+API
>
> Though on the surface it may not appear that different, I've written it
> up as though GeoServer has a single web interface - not separating out
> an API from a user interface. There are also a few new ideas in there
> that I'm interested to hear feedback on.
>
> 1) I think it would be cool to think big picture style about how a data
> config API and user managment API fit in to the rest of GeoServer's web
> interface. Largely, this is about tucking things in the /api path -
> which suggests that there should be a different interface for
> non-programmed users. I like the idea that GeoServer exposes a number
> of resources through one interface on the web - and that the same
> interface is useful for humans and machines.

I like the idea of a single consistent REST api that goes from
configuration to actual data.
A user interface based on the same structure (that is, one that
is completely based in rest style, that is, simply return back HTML
as the default resource encoding) may be appealing, thought I'm not sure
how you would handle in REST style a few typical use cases:
- per feature type service related configurations.
   As an example of service specific knob, have a per feature type
   maxFeature setting that is WFS specific. Or the styles, that in fact
   are WMS specific.
   As we add services to GeoServer, the list of service specific knobs
   will increase. We could have a special subpage for each, like:
   {featureType}/services/{serviceName} but wouldn't it make things a
   little too verbose (navigation wise)

Another option would be to have the per-service options be their own
hierarchy, like
/services/{serviceName}/workspaces/{workspace}/layer/{layername}. I also feel
like it would be all right to have the 'html' output format be kind of a
special case, so that /workspaces/{workspace}/layer/{layername}.html could
have multiple forms or something so that it acts as a frontend for all of
these different resources.

- custom per data store creation pages. The current approach uses
   a generic form that no one really likes, it would be better to have
   custom, specific configuration pages used to create a new datastore.
   But there would be aids to create a new datastore, they would not
   be resources per se. How do you handle that?

Perhaps we could special-case this as well. I still feel that having
alternate 'wizard' and 'raw options' ui's would be useful, so we could
perhaps have multiple html output formats. Or it could be a query parameter
or something (like ?advanced=true, defaulting to false).

Btw, I noticed now that the proposal page does not have the concept
of a datastore, that is, one of a connection to a database, remote wfs
server, or a file (and the same goes for coverages).

Sure it does, each of these would be a workspace. Well, except for the files.
I think the plan is to have a physical folder look like a workspace as well,
since shapefiles etc. each only contain one layer. At least, I hope so,
since I've already got some code in RESTconfig to make it look that way

If you're following the mapserver approach, you'd have to specify over
and over again the same connection params for each layers, which I'd
find quite a bit cumbersome.
The current approach is cumbersome in its own way too, since you have
to manually go and register layers one by one, it would be a lot better
if after registering the datastore you had a list of all the possible
feature types, and you cherry pick the ones you want to expose
via GeoServer. Resource wise, it would be nice to have the following:

/geoserver/workspaces/<wsName>/datastores/{dsName}/<ftName>

that is, a way to expose feature types that are there, but aren't
still made available thru GeoServer. We could use a request to
turn each ftName on and off, when you turn it on, it becomes an
available layer in:

/geoserver/workspaces/<name>/layers

I've talked to cholmes and tschaub a bit about having
the /geoserver/workspaces/{name}/layers resource return a slightly different
result for users logged in as administrators, that has two lists, one of
configured layers and one of unconfigured, but available layers. Of course,
if actual W*S requests can only be made against layers that have been
published (ie, exposed by configuring them to appear under /maps) then maybe
the distinction between configured layers and unconfigured ones is not very
useful?

Same would happen for coverages and coverage stores.
The thing is, we'd need to make datastores and coveragestores
paths visible only to admin user types, whilst the layers part
would be visible to everybody (as a REST style "capabilities" offering).

Cheers
Andrea

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplac
e _______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4040,47eb6115187215409313003!

[Justin_Deoliveira]

My thoughts on this one are that we should treat namespace as an attribute on layer, and not as a grouping of layers. I also dont think namespace makes it into the "workspace" hierachy either. I would like to see it be something seperate:

GET geoserver/namespaces/ (get a list of namespaces)
GET geoserver/namespaces/<namespace>/layers (get a list of layers in a namespace)

Like datastore and featuretype, namespace is something abstract and hard for people to understand. It also places a restriction on us to force the use of it when referencing a wms layer.

While they are useful for preventing layer name collisions, and something we need for WFS, i think we can keep them around without ramming it down people throats.

Andrea Aime wrote:

Tim Schaub ha scritto:
...

2) I think it makes sense to have one set of resources related to configuring data (workspaces) and one set related to publishing layers (maps). The name "workspaces" captures the meaning of a resource for the configuration of layer collections a bit better for me than "folders" - where that collection can be a database connection, a disk based folder, or a group of remote layers via an OGC service. Workspaces (places to configure data typically restricted to an administrator) are distinguished from maps (collections of published layers typically accessible by an anonymous user).

I'm having troubles matching worskpaces and maps to what we have today.

Workspace seems to recall a little the idea of namespace, and a little
the idea of a data directory.

Namespaces are today the only organisational structure we have in GeoServer, each datastore and coveragestore is placed under a certain
namespace, and will be soon used to drive per layer security restrictions.
Unfortunately namespace by itself is not strong enough to replace
the concept of a data directory, since we don't have a per namespace
service and server configuration.
This leads users to deploy geoserver multiple times, in order to have
specific service configurations, like, for example, bilinear interpolation for a group of wms coverage layers, but not for the others, or have WFS-T enabled in one case but not on the other.

If we had the possibility to state a per namespace service level
configuration as well, it would make it possible to treat namespaces
as the organisational structure you called workspaces.

Also mind that namespaces are needed for wfs anyways, so we cannot
eliminate them. What I'm proposing is to make them strong enough
to represent the concept of workspace, a container that has
certain layers and certain service and server configurations: think
wms interpolation, antialiasing, logging level, but also set of users,
security policies and the like.

The major beneficial side effect of this would be that we won't
need anymore to deploy multiple geoserver just to handle different
user groups (think an ASP that wants to offer GeoServer hosting to
multiple customers).

Which brings me to the map concept, that seems to be modelled
after the MapServer mapfile. At first I thought it was like a layer
group (a single name used in wms calls to signify a stack of layers with
pre-confired styles), but looking at the REST api, it's not... or is it?

I'm wondering if we really need this concept or not. The workspaces
already provide a way to organise layers, why do we need a separate
map concept? Styles and projections could be very well attached
to workspace layers (which is what is happening today).

The only reason I see that may drive people to use the map concept is
to be able to lock down what styles you may use for a certain layer,
and have different lock downs for different users.
But it seems to me this defeats the purpose of an SDI as a way to
let people freely look at the data, combine it in different ways.
It would be ineffective too unless we disallow people to provide
the &sld parameter.
Finally, it would require quite a bit of changes in WMS code as well since, as far as I understand it, each "map" would have to be represented by a separate WMS service no?

I'm not saying no to the idea, but I'm wondering "where's the meat" in the map concept.

Cheers
Andrea

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4007,47eb6581193295409313003!

--
Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com

[David_Winslow]

On Thursday 27 March 2008 05:14:13 Andrea Aime wrote:

Tim Schaub ha scritto:
...

> 2) I think it makes sense to have one set of resources related to
> configuring data (workspaces) and one set related to publishing layers
> (maps). The name "workspaces" captures the meaning of a resource for
> the configuration of layer collections a bit better for me than
> "folders" - where that collection can be a database connection, a disk
> based folder, or a group of remote layers via an OGC service.
> Workspaces (places to configure data typically restricted to an
> administrator) are distinguished from maps (collections of published
> layers typically accessible by an anonymous user).

I'm having troubles matching worskpaces and maps to what we have today.

Workspace seems to recall a little the idea of namespace, and a little
the idea of a data directory.

As I understand it, workspaces are meant to be a generalization of
data/coveragestores. They are resources that contain map layers, whether
they be raster or vector. This grouping is by the technical implementation;
that is, a workspace might correspond to a postgis database connection or a
folder full of files on the filesystem, but it would not mix them.

Maps are basically configurable views into the workspaces, that let you group
layers regardless of implementation, and provide a means of working with
those groups as needed (setting styles, security settings, whatever.) I
think we'd like to have an inheritance scheme for configuring these, ie, you
can set a particular default style for a map and override it for a specific
layer. As I understand it, you wouldn't really be able to treat maps as a
single layer the way you can layergroups in today's setup.

short version: workspace == (data|coverage)store, map == namespace

I am not really sure what would be analogous to layergroups in the proposed
API, perhaps it would make sense to have a new type of workspace that can
contain layergroups, and then each layer in that workspace would have a
single configuration option that was a list of other layers.

Namespaces are today the only organisational structure we have in
GeoServer, each datastore and coveragestore is placed under a certain
namespace, and will be soon used to drive per layer security restrictions.
Unfortunately namespace by itself is not strong enough to replace
the concept of a data directory, since we don't have a per namespace
service and server configuration.
This leads users to deploy geoserver multiple times, in order to have
specific service configurations, like, for example, bilinear
interpolation for a group of wms coverage layers, but not for the
others, or have WFS-T enabled in one case but not on the other.

If we had the possibility to state a per namespace service level
configuration as well, it would make it possible to treat namespaces
as the organisational structure you called workspaces.

Also mind that namespaces are needed for wfs anyways, so we cannot
eliminate them. What I'm proposing is to make them strong enough
to represent the concept of workspace, a container that has
certain layers and certain service and server configurations: think
wms interpolation, antialiasing, logging level, but also set of users,
security policies and the like.

The major beneficial side effect of this would be that we won't
need anymore to deploy multiple geoserver just to handle different
user groups (think an ASP that wants to offer GeoServer hosting to
multiple customers).

Which brings me to the map concept, that seems to be modelled
after the MapServer mapfile. At first I thought it was like a layer
group (a single name used in wms calls to signify a stack of layers with
pre-confired styles), but looking at the REST api, it's not... or is it?

I'm wondering if we really need this concept or not. The workspaces
already provide a way to organise layers, why do we need a separate
map concept? Styles and projections could be very well attached
to workspace layers (which is what is happening today).

The only reason I see that may drive people to use the map concept is
to be able to lock down what styles you may use for a certain layer,
and have different lock downs for different users.
But it seems to me this defeats the purpose of an SDI as a way to
let people freely look at the data, combine it in different ways.
It would be ineffective too unless we disallow people to provide
the &sld parameter.
Finally, it would require quite a bit of changes in WMS code as well
since, as far as I understand it, each "map" would have to be
represented by a separate WMS service no?

I'm not saying no to the idea, but I'm wondering "where's the meat" in
the map concept.

Cheers
Andrea

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplac
e _______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4040,47eb6580193271804284693!

[David_Winslow]

On Thursday 27 March 2008 05:23:52 Andrea Aime wrote:

Tim Schaub ha scritto:
...

> 4) I also don't know squat about auth/auth in GeoServer, but it seems to
> me like it would be pretty handy to expose a "groups" resource (and/or
> "roles") next to the users resource. Then every other resource (with
> the exception of individual features perhaps) could have a "permissions"
> resource that described how each group (or role) was limited by verb
> (get, post, put, delete) - with an option for inheritance from a parent
> resource. I may be naive, but that seems like a pretty nice way to
> control security.

At the moment GeoServer security is pretty lame, it's based only on
the service and method, that is, you can say only certain users (roles
in fact, RBAC security: http://en.wikipedia.org/wiki/RBAC) can access
a certain service method, like in:
WFS.Transaction=WFS_T_USER,ADMINISTRATOR
where you state that the WFS Transaction method can be called only
by users having either WFS_T_USER or ADMINISTRATOR role.

If things go well (sponsoring wise) we should be able to have namespace
and layer based security, that is, something like:
topp.*.r=TOPP_USER
topp.*.w=TOPP_USER
topp.states.w=STATES_WRITER
stating that every layer in the topp namespace can be accessed only by
users that have the TOPP_USER role, with an override for topp:states
saying that only users with users with STATES_WRITER role can modify
the states layer.

There is no plan to have per feature or per property access control
(think restricted areas, or data columns that only certain users can
see) at the moment.

REST wise, this could be represented as:
/worskspaces/<wsname>/layers/<layer>/permissions/read
/worskspaces/<wsname>/layers/<layer>/permissions/write
/worskspaces/<wsname>/permissions/read
/worskspaces/<wsname>/permissions/write
(where each read/write is a list of roles)
and as
/worskspaces/<wsname>/users/<user>/roles

Cheers
Andrea

Just to clarify, my current conception of the security configuration for the
new REST API is to have permissions for the /maps hierarchy apply to the map
data, and permissions for the /workspaces hierarchy would apply to
configuration (maybe you have a set of users who you'd like to allow
uploading shapefiles, but you want to keep them from borking your postgres
connection settings). Each resource, not just the named ones would have
a /permissions child, like so:
/workspaces/permissions - allow you to restrict resource creation/deletion
/workspaces/{name}/permissions - allow to restrict modification of resource
{name}

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplac
e _______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4040,47eb67b8196495409313003!

[Andrea_Aime3]

Justin Deoliveira ha scritto:

My thoughts on this one are that we should treat namespace as an attribute on layer, and not as a grouping of layers.

Technically namespace is an attribute of the datastore, not of the layer, and there's not much we can do about it, unless we do change
the datastores in gt2.
So if workspace == datastore, then the structure has to be

{namespace}/{workspace}/...

because of how gt2 is done.

I also dont think namespace makes it into the "workspace" hierachy either. I would like to see it be something seperate:

GET geoserver/namespaces/ (get a list of namespaces)
GET geoserver/namespaces/<namespace>/layers (get a list of layers in a namespace)

Like datastore and featuretype, namespace is something abstract and hard for people to understand. It also places a restriction on us to force the use of it when referencing a wms layer.

This has never happened, and it's not happening today either, unless
a bug is happening. You can reference layers in the default namespace
without prefixing them with the namespace prefix. If you start using
multiple namespace the prefix is needed since the same layer name can be
used in the both of them.
See for example:

http://geo.openplans.org:8080/geoserver/wms?bbox=-127.61950064999999,23.7351786,-64.08177035,50.592523400000005&styles=&Format=application/openlayers&request=GetMap&layers=states&width=800&height=317&srs=EPSG:4326

(notice the layer name, it's states, not topp:states).

Cheers
Andrea

[Andrea_Aime3]

David Winslow ha scritto:
...

Just to clarify, my current conception of the security configuration for the new REST API is to have permissions for the /maps hierarchy apply to the map data, and permissions for the /workspaces hierarchy would apply to configuration (maybe you have a set of users who you'd like to allow uploading shapefiles, but you want to keep them from borking your postgres connection settings). Each resource, not just the named ones would have a /permissions child, like so:
/workspaces/permissions - allow you to restrict resource creation/deletion
/workspaces/{name}/permissions - allow to restrict modification of resource {name}

That's ok, but in the proposal we're making for that security bid we
don't have the datastore concept, we have the namespace one. If we ever
manage to get that one, where do we develop it? And how do we justify
swiching from namespace controlled security to datastore controlled
security?

Cheers
Andrea

[Justin_Deoliveira]

Technically namespace is an attribute of the datastore, not of the layer, and there's not much we can do about it, unless we do change
the datastores in gt2.
So if workspace == datastore, then the structure has to be

{namespace}/{workspace}/...

because of how gt2 is done.

That is how it is coded in geotools but is that how we have to make it available in geoserver? I could be wrong but I would think no. Some of the geotools datastores don't even respect the namespace parameter that is passed in, and the feature types just end up being in the gml namespace.

Also, I don't think the model we expose via our REST api has to expose the same structure that we use internally. On the contrary i think it should hide much of it.

I also dont think namespace makes it into the "workspace" hierachy either. I would like to see it be something seperate:

GET geoserver/namespaces/ (get a list of namespaces)
GET geoserver/namespaces/<namespace>/layers (get a list of layers in a namespace)

Like datastore and featuretype, namespace is something abstract and hard for people to understand. It also places a restriction on us to force the use of it when referencing a wms layer.

This has never happened, and it's not happening today either, unless
a bug is happening. You can reference layers in the default namespace
without prefixing them with the namespace prefix. If you start using
multiple namespace the prefix is needed since the same layer name can be
used in the both of them.
See for example:

http://geo.openplans.org:8080/geoserver/wms?bbox=-127.61950064999999,23.7351786,-64.08177035,50.592523400000005&styles=&Format=application/openlayers&request=GetMap&layers=states&width=800&height=317&srs=EPSG:4326

(notice the layer name, it's states, not topp:states).

Right, but the key here is "default namespace", lets say the layer is in a non default and you provide it non-prefixed (even though there is just one), until a recent patch i submitted it would not work.

I also think that once we have the idea of "maps", we dont need namespaces anymore to prevent naming conflicts. A layer name is unique within a particular map, which i think is a reasonable restriction.

Cheers
Andrea

!DSPAM:4007,47ebc0b412853668746562!

--
Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com

[David_Winslow]

On Thursday 27 March 2008 11:46:11 Andrea Aime wrote:

David Winslow ha scritto:
...

> Just to clarify, my current conception of the security configuration for
> the new REST API is to have permissions for the /maps hierarchy apply to
> the map data, and permissions for the /workspaces hierarchy would apply
> to configuration (maybe you have a set of users who you'd like to allow
> uploading shapefiles, but you want to keep them from borking your
> postgres connection settings). Each resource, not just the named ones
> would have a /permissions child, like so:
> /workspaces/permissions - allow you to restrict resource
> creation/deletion /workspaces/{name}/permissions - allow to restrict
> modification of resource {name}

That's ok, but in the proposal we're making for that security bid we
don't have the datastore concept, we have the namespace one. If we ever
manage to get that one, where do we develop it? And how do we justify
swiching from namespace controlled security to datastore controlled
security?

We can probably punt on the administrative side security settings for now (ie,
just hard code it to look for ROLE_ADMINISTRATOR like now). Security
settings on the maps/ hierarchy should be analogous to namespace-based access
control to the map data itself, the workspaces hierarchy is just for
configuration.

-David

Cheers
Andrea

!DSPAM:4040,47ebc14714207082231907!

[Andrea_Aime3]

David Winslow ha scritto:

On Thursday 27 March 2008 05:14:13 Andrea Aime wrote:

Tim Schaub ha scritto:
...

2) I think it makes sense to have one set of resources related to

configuring data (workspaces) and one set related to publishing layers
(maps). The name "workspaces" captures the meaning of a resource for
the configuration of layer collections a bit better for me than
"folders" - where that collection can be a database connection, a disk
based folder, or a group of remote layers via an OGC service.
Workspaces (places to configure data typically restricted to an
administrator) are distinguished from maps (collections of published
layers typically accessible by an anonymous user).

I'm having troubles matching worskpaces and maps to what we have today.

Workspace seems to recall a little the idea of namespace, and a little
the idea of a data directory.

As I understand it, workspaces are meant to be a generalization of data/coveragestores. They are resources that contain map layers, whether they be raster or vector. This grouping is by the technical implementation; that is, a workspace might correspond to a postgis database connection or a folder full of files on the filesystem, but it would not mix them.

Ok, I get it now, I was confused by the name, which reminded me of something more general than just a datastore.

In this case to make the concept feasible we need two new elements in GeoServer:
* a directory datastore that list as feature types the files contained
   in a folder
* a real coverage store abstraction, that today we lack completely
   (talks about it never went past the "would be nice to have one"
    point).

Otherwise we'd end up with tens of workspaces, one for each shapefile
and coverage we connect to.

Maps are basically configurable views into the workspaces, that let you group layers regardless of implementation, and provide a means of working with those groups as needed (setting styles, security settings, whatever.) I think we'd like to have an inheritance scheme for configuring these, ie, you can set a particular default style for a map and override it for a specific layer. As I understand it, you wouldn't really be able to treat maps as a single layer the way you can layergroups in today's setup.

short version: workspace == (data|coverage)store, map == namespace

I see... unfortunately this is not technically feasible today since namespace is an attribute of the datastore, you cannot change it at will
for single layers, or have the same layer appear under different namespaces.

We can work this out by wrapping each feature type feauturesource and change the namespace on the fly... thought it might become a little
confusing code wise.

I am not really sure what would be analogous to layergroups in the proposed API, perhaps it would make sense to have a new type of workspace that can contain layergroups, and then each layer in that workspace would have a single configuration option that was a list of other layers.

eek... I'm starting too see too many level of indirections, or I'm just
tired today?

Cheers
Andrea

[Andrea_Aime3]

Justin Deoliveira ha scritto:

Technically namespace is an attribute of the datastore, not of the layer, and there's not much we can do about it, unless we do change
the datastores in gt2.
So if workspace == datastore, then the structure has to be

{namespace}/{workspace}/...

because of how gt2 is done.

That is how it is coded in geotools but is that how we have to make it available in geoserver? I could be wrong but I would think no.

It certainly makes code easier than trying to fake what's not there.
That said we have already quite a bit of indirection levels, adding one
more won't kill us.
It just seems unecessary to me. We could just change the wording
and call namespace workspaces,folders,bags,whatever and keep the tech side simple.

Some of the geotools datastores don't even respect the namespace parameter that is passed in, and the feature types just end up being in the gml namespace.

This is a bug that has to be fixed, not a good reason to kill namespaces
in the first place imho

Right, but the key here is "default namespace", lets say the layer is in a non default and you provide it non-prefixed (even though there is just one), until a recent patch i submitted it would not work.

I also think that once we have the idea of "maps", we dont need namespaces anymore to prevent naming conflicts. A layer name is unique within a particular map, which i think is a reasonable restriction.

Since we're a WFS server namespaces can't really be removed or completely hidden, you have to be able to reference the same layer the same way across services (using wms and wfs at the same time is not uncommon).

OGC tests ask for multiple namespaces no? And if I understand correctly,
the map would be associated to a single wfs,wcs,wms endpoint, meaning
that we would have a single capabilities document per map.
What if OGC one day wakes up and asks that we do prove we can support
the same layer name in different namespaces (say, for WFS 1.2 cite tests)? For them it would be a legitimate request, and we would not be
able to support that anymore.

Cheers
Andrea

[Andrea_Aime3]

David Winslow ha scritto:

On Thursday 27 March 2008 11:46:11 Andrea Aime wrote:

David Winslow ha scritto:
...

Just to clarify, my current conception of the security configuration for
the new REST API is to have permissions for the /maps hierarchy apply to
the map data, and permissions for the /workspaces hierarchy would apply
to configuration (maybe you have a set of users who you'd like to allow
uploading shapefiles, but you want to keep them from borking your
postgres connection settings). Each resource, not just the named ones
would have a /permissions child, like so:
/workspaces/permissions - allow you to restrict resource
creation/deletion /workspaces/{name}/permissions - allow to restrict
modification of resource {name}

That's ok, but in the proposal we're making for that security bid we
don't have the datastore concept, we have the namespace one. If we ever
manage to get that one, where do we develop it? And how do we justify
swiching from namespace controlled security to datastore controlled
security?

We can probably punt on the administrative side security settings for now (ie, just hard code it to look for ROLE_ADMINISTRATOR like now). Security settings on the maps/ hierarchy should be analogous to namespace-based access control to the map data itself, the workspaces hierarchy is just for configuration.

This is fine, but does not work unless we change how namespaces are
handled in GeoServer and GeoTools somehow. If Justin interpretation is
correct, map is something different than namespace.

Cheers
Andrea

[Justin_Deoliveira]

It certainly makes code easier than trying to fake what's not there.
That said we have already quite a bit of indirection levels, adding one
more won't kill us.
It just seems unecessary to me. We could just change the wording
and call namespace workspaces,folders,bags,whatever and keep the tech side simple.

Indirection? I fail to see how moving namespace from a datastore attribute to a feature type attribute introduces another level of indirection.

Some of the geotools datastores don't even respect the namespace parameter that is passed in, and the feature types just end up being in the gml namespace.

This is a bug that has to be fixed, not a good reason to kill namespaces
in the first place imho

I did not say we should kill namespace. All i said is that we should not expose it in the resource hierarchy of our rest structure because its confusing. I am 100% for the grouping of layers by namespaces, we need it for wfs.

And is it a bug? I agree that namespace serves the purposes of uniquely identifying a layer. What I am saying is that we should use the idea of a "map" to do it in WMS. WFS is written in terms of namespace so i am 100% fine making people continue to use it there.

Right, but the key here is "default namespace", lets say the layer is in a non default and you provide it non-prefixed (even though there is just one), until a recent patch i submitted it would not work.

I also think that once we have the idea of "maps", we dont need namespaces anymore to prevent naming conflicts. A layer name is unique within a particular map, which i think is a reasonable restriction.

Since we're a WFS server namespaces can't really be removed or completely hidden, you have to be able to reference the same layer the same way across services (using wms and wfs at the same time is not uncommon).

Thanks, but I already understand the concept of a namespace in WFS.

OGC tests ask for multiple namespaces no? And if I understand correctly,
the map would be associated to a single wfs,wcs,wms endpoint, meaning
that we would have a single capabilities document per map.
What if OGC one day wakes up and asks that we do prove we can support
the same layer name in different namespaces (say, for WFS 1.2 cite tests)? For them it would be a legitimate request, and we would not be
able to support that anymore.

Yes they do, and again i am not saying that we get rid of namespace. See above comment.

Cheers
Andrea

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4007,47ebc6a743451096210785!

--
Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com

[Tim_Schaub2]

Thanks for the thoughts Andrea.

I'll respond in even smaller bits

Andrea Aime wrote:

Btw, I noticed now that the proposal page does not have the concept
of a datastore, that is, one of a connection to a database, remote wfs
server, or a file (and the same goes for coverages).

If you're following the mapserver approach, you'd have to specify over
and over again the same connection params for each layers, which I'd
find quite a bit cumbersome.

See workspace.

"A workspace is a folder-like collection of layers. A workspace can represent a database connection, a folder on disk (containing shapefiles, GeoTIFFs, etc.), or a collection of remote OGC services (cascading WMS for example)."

The current approach is cumbersome in its own way too, since you have
to manually go and register layers one by one, it would be a lot better
if after registering the datastore you had a list of all the possible
feature types, and you cherry pick the ones you want to expose
via GeoServer. Resource wise, it would be nice to have the following:

/geoserver/workspaces/<wsName>/datastores/{dsName}/<ftName>

My vision is that you create a new workspace (say a database connection) doing something like the following - excuse the shorthand:

Request:
POST /geoserver/workspaces/
<body contains workspace config>

Response:
<workspace url>

Then you want to see what layers (term layers used flexibly here) the workspace has:

Request:
GET /geoserver/workspaces/<name>/layers (name from <workspace url> above)

Response:
<Depending on exactly what format you asked for, the response will vary. A nice response in my mind would be a structure (optionally json) that listed the already configured layers and those that had not yet been configured. This would, of course, be a big list if you had a lot of tables in a database (for example). Something like the following makes a nice list in my mind:

[{
     "name": string,
     "enabled": boolean,
     "properties": object
}, {
     ...
}]

Or something like that.>

that is, a way to expose feature types that are there, but aren't
still made available thru GeoServer. We could use a request to
turn each ftName on and off, when you turn it on, it becomes an
available layer in:

/geoserver/workspaces/<name>/layers

Yeah, don't know what makes the most sense to people. One option would be to immediately expose a resource for each layer in a workspace (to the extent that you can determine what makes a layer). To enable a layer that is not currently enabled, you would put an updated configuration to the layer resource url

Request:
PUT /geoserver/workspaces/<name>/layers/<name>
{
     "enabled": true,
     ...
}

Another option would be to not call these things layers until they were enabled explicitly. In this case, you could get hints about what could be configured as a layer by issuing a GET request directly to the named workspace. The response could include a list of things in the workspace that are candidates for configuration as layers.

Same would happen for coverages and coverage stores.
The thing is, we'd need to make datastores and coveragestores
paths visible only to admin user types, whilst the layers part
would be visible to everybody (as a REST style "capabilities" offering).

Yeah, using permissions everywhere. A typical configuration would only show enabled workspace layers to users not in an admin group. Or perhaps only users in the admin group would be able to see workspaces. In any case, I think you have what you need. I don't differentiate between feature types and coverages because I think the term layer is more sensible (the response from a request to get layer information can always include detail about the type of store that the layer is derived from).

Tim

Cheers
Andrea

!DSPAM:4033,47eb6105187021439371379!

[Andrea_Aime3]

Justin Deoliveira ha scritto:

It certainly makes code easier than trying to fake what's not there.
That said we have already quite a bit of indirection levels, adding one
more won't kill us.
It just seems unecessary to me. We could just change the wording
and call namespace workspaces,folders,bags,whatever and keep the tech side simple.

Indirection? I fail to see how moving namespace from a datastore attribute to a feature type attribute introduces another level of indirection.

Well, since geotools datastore do not attach namespaces to feature types
but to the datatsore as whole, I thought you might have wanted to wrap
the FeatureSource to change the namespace of each single layer on the fly. That would be an extra indirection, an extra wrapping.
How did you intended to address this?

And is it a bug? I agree that namespace serves the purposes of uniquely identifying a layer. What I am saying is that we should use the idea of a "map" to do it in WMS. WFS is written in terms of namespace so i am 100% fine making people continue to use it there.

It is a bug since we pass in a namespace parameter into the datastore
at creation time to have feature types carry a certain namespace.
If the datastore does not respect it, it looks like a bug to me.
If you passed in a username and the datastore ignored it using a hardcoded one, what would you call that? A feature?

Right, but the key here is "default namespace", lets say the layer is in a non default and you provide it non-prefixed (even though there is just one), until a recent patch i submitted it would not work.

I also think that once we have the idea of "maps", we dont need namespaces anymore to prevent naming conflicts. A layer name is unique within a particular map, which i think is a reasonable restriction.

Since we're a WFS server namespaces can't really be removed or completely hidden, you have to be able to reference the same layer the same way across services (using wms and wfs at the same time is not uncommon).

Thanks, but I already understand the concept of a namespace in WFS.

Never implied you did not, you're reading too much in what I write
It's just that I don't understand.
You said: "I also think that once we have the idea of "maps", we dont need namespaces anymore to prevent naming conflicts"
Which makes me think you want to make it working without namespace
prefixes the default. Yet we do use namespace prefixes in wfs.
So if you use both wfs and wms, you either make an unprefixed
call to wms and a prefixed call to wfs? That's what confusing me.

Cheers
Andrea