[Geoserver-devel] Concept for online tests

I developed an user/group/role backend store using jdbc and have a test case using H2. I want to test against other databases as well, but I do not find a proper description in the developer guide about online tests.

Should I go the same way as with geotools online tests, using a fixture ?.

Cheers
Christian

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

On Wed, Jun 29, 2011 at 1:16 PM, <christian.mueller@anonymised.com> wrote:

I developed an user/group/role backend store using jdbc and have a
test case using H2. I want to test against other databases as well,
but I do not find a proper description in the developer guide about
online tests.

Should I go the same way as with geotools online tests, using a fixture ?.

I think so, yeah.
If you look in the extensions/wfsv module there are tests that do need
a postgis database to run and they basically follow that same principle

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

In the meantime I have implemented xml and jdbc stores for user/group and role backend stores. The questions is how to integrate the configuration into gesoerver.

1) Work with with files and directories within GEOSERVER_DATA_DIR/security. I would use Java Properties serialized in XML.

2) Create CatalogInfo Objects and integrate the these configurations into the catalog.

It should be possible to use REST to manage these backend stores.

Or is it easier to start with 1) and defer 2).

Cheers
Christian

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

On Wed, Jun 29, 2011 at 4:12 PM, <christian.mueller@anonymised.com…1326…> wrote:

In the meantime I have implemented xml and jdbc stores for user/group and role backend stores. The questions is how to integrate the configuration into gesoerver.

  1. Work with with files and directories within GEOSERVER_DATA_DIR/security. I would use Java Properties serialized in XML.

  2. Create CatalogInfo Objects and integrate the these configurations into the catalog.

It should be possible to use REST to manage these backend stores.

Or is it easier to start with 1) and defer 2).

Let’s start with 2. Nothing prevents you from using the CatalogInfo interfaces, but what would be
the reason?
Security has been so far a orthogonal concern to data and service configuration and mostly
transparent, that is, most of the code does not know there even is a security layer, I believe
it should stay that way.
CatalogInfo objects are instead known to the whole application, while you can in theory change
the backend implementation everything works and plays against the CatalogInfo sub-interface,
they define what we manage and what can be known about features, coverages and the like.

The security subsytem is a different kind of beast, what we can say we know about users
is often just the user name: different implementations of the authentication/authorization
subsytem could have or not explicit roles, roles could be, or not, plain strings, for all
we know the “thing” that describes the user access rights could be a complex data structure.
Spring Security is very generic about users for this reason I believe, a Authentication
has credentials (a generic object), authorities (a list of objects) and details (guess what,
an object).

Now, of course the default implementation of the security subsystem needs to know
something about the users, our current one says that a user has a list of roles
which are strings for example.
And a new default implementation would be the same, but the notion that a user
has a name and roles that are strings should be restricted to the specifics of that
implementation (its management, its GUI, its rest interface),
without leaking in any other part of GeoServer.
So yeah, it could be made of CatalogInfo objects if you wish, I’m just worried
that by seeing them as CatalogInfo there would be temptation to have them known
in other parts of the system, to have the leak around.

Now, going back to persistence and REST. Those two are not necessarily related, but
it helps to have the same persistence format, or at least a configurable one that
can be bent to both usages, for those two.
I’m not sure what the rationale is for using property files in xml format though?
As far as I know the only advantage over plain property files is that you can store
multibyte chars in them?
Another curiosity is, what about the network of security objects to be stored
in 3-5 XML files that we were talking about last time the topic popped up?
What is the current implementation plan?

Anyways, if you want to go property files I have nothing specific against them,
though using XStream would have given you uniformity with how we already
manage configuration and rest resource encoding for the catalog, also with interesting
bits on how to have the bits stored on disk relate to each other via ids, whilst the
network format uses atom links in their place to ensure connectivity.

Did you already think about a REST api for managing users and permissions?
What would be the resources, their granularity and representation?

Cheers
Andrea

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Puuh, long mail, answers inside

Zitat von Andrea Aime <andrea.aime@anonymised.com>:

On Wed, Jun 29, 2011 at 4:12 PM, <christian.mueller@anonymised.com> wrote:

In the meantime I have implemented xml and jdbc stores for user/group and
role backend stores. The questions is how to integrate the configuration
into gesoerver.

1) Work with with files and directories within GEOSERVER_DATA_DIR/security.
I would use Java Properties serialized in XML.

2) Create CatalogInfo Objects and integrate the these configurations into
the catalog.

It should be possible to use REST to manage these backend stores.

Or is it easier to start with 1) and defer 2).

Let's start with 2. Nothing prevents you from using the CatalogInfo
interfaces, but what would be
the reason?

One reason is that I need an object model to develop against, especially for the GUI and the REST API. I am talking here only about configuration parameters like the jdbc connect params I need for the jdbc backend for /user/groups/roles. Other config options are schema validating yes/no for the xml backend store, a boolean property "reentrant" which causes the creation of a locking wrapper and so on.

Other possibilities would be to work with individual java BeainInfo objects for each backend store ore using properties, but I fear this will produce more code on the GUI and REST. I dislike both of them.

Conclusion: I would like to have a proper object model for the configuration issues.

Security has been so far a orthogonal concern to data and service
configuration and mostly
transparent, that is, most of the code does not know there even is a
security layer, I believe
it should stay that way.

Yep, it should, "orthogonal" is a very good expression.

CatalogInfo objects are instead known to the whole application, while you
can in theory change
the backend implementation everything works and plays against the
CatalogInfo sub-interface,
they define what we manage and what can be known about features, coverages
and the like.

What about using the CatalogInfo interface without registering this objects into the catalog ?. We can discuss integration if you see the whole thing in action ?

The security subsytem is a different kind of beast, what we can say we know
about users
is often just the user name: different implementations of the
authentication/authorization
subsytem could have or not explicit roles, roles could be, or not, plain
strings, for all
we know the "thing" that describes the user access rights could be a complex
data structure.
Spring Security is very generic about users for this reason I believe, a
Authentication
has credentials (a generic object), authorities (a list of objects) and
details (guess what,
an object).

Now, of course the default implementation of the security subsystem needs to
know
something about the users, our current one says that a user has a list of
roles
which are strings for example.
And a new default implementation would be the same, but the notion that a
user
has a name and roles that are strings should be restricted to the specifics
of that
implementation (its management, its GUI, its rest interface),
without leaking in any other part of GeoServer.
So yeah, it could be made of CatalogInfo objects if you wish, I'm just
worried
that by seeing them as CatalogInfo there would be temptation to have them
known
in other parts of the system, to have the leak around.

Short outline: I have 4 main classes
GeoserverUser implementing Spring UserDetails
GeoserverUserGroup which is new
GeoserverGrantedAuthority inherits Spring GrantedAuthority

Additionally, there is an Interface GeoserverSecuritySystem which implements
Spring UserDetailsService, a singleton is created by Spring injection.

I took care that my classes behave like expected from Spring (studying the source code), only the calculation of UserDetails.getAuthorities is more complex, but this is not of interest to the spring security architecture.

Now, going back to persistence and REST. Those two are not necessarily
related, but
it helps to have the same persistence format, or at least a configurable one
that
can be bent to both usages, for those two.

Yeah, I want the configuration objects to behave like other geoserver configuration objects. Individual files for backend implementations are independent. As an example, for the jdbc backend I need files for the DDL and the DML statements, no plan to make CatalogInfo objects for these files.

I'm not sure what the rationale is for using property files in xml format
though?
As far as I know the only advantage over plain property files is that you
can store
multibyte chars in them?

I like the xml approach because of name spaces. As an example, I could combine the DDL xml file and the DML xml file into one xml file using 2 name spaces. If users dislike the xml property files, I could change that.

Another curiosity is, what about the network of security objects to be
stored
in 3-5 XML files that we were talking about last time the topic popped up?
What is the current implementation plan?

There are 2 xml backends, one for user/group info and one for roles (GrantedAuthorities). The default deployment will have 2 xml files.

Anyways, if you want to go property files I have nothing specific against
them,
though using XStream would have given you uniformity with how we already
manage configuration and rest resource encoding for the catalog, also with
interesting
bits on how to have the bits stored on disk relate to each other via ids,
whilst the
network format uses atom links in their place to ensure connectivity.

Not sure what your are meaning here, I did not invest much time how CatalogInfo works. I will start at DataStoreInfo to understand the architecture.

Did you already think about a REST api for managing users and permissions?
What would be the resources, their granularity and representation?

In a first step, I only want to handle the configuration using REST, but at the end of the day, I plan to offer the same possibilities for the GUI and the REST API.

Cheers
Andrea

Summary:
I will go on developing an object model for configuration issues based on CatalogInfo. I want to defer/avoid altering the Catalog interface, perhaps I implement something similar for the security subsystem.

I fear that I will produce a "monster" patch, so I want to find some boundaries
not preventing further integration in the future.

I hope my English is clear enough, one of my sons insists on going to the playground, no time to reread, no chance at the moment :slight_smile:

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

On Sat, Jul 2, 2011 at 5:43 PM, <christian.mueller@anonymised.com.1326…> wrote:

Let’s start with 2. Nothing prevents you from using the CatalogInfo
interfaces, but what would be
the reason?

One reason is that I need an object model to develop against, especially for the GUI and the REST API. I am talking here only about configuration parameters like the jdbc connect params I need for the jdbc backend for /user/groups/roles. Other config options are schema validating yes/no for the xml backend store, a boolean property “reentrant” which causes the creation of a locking wrapper and so on.

Other possibilities would be to work with individual java BeainInfo objects for each backend store ore using properties, but I fear this will produce more code on the GUI and REST. I dislike both of them.

Conclusion: I would like to have a proper object model for the configuration issues.

Sure, a proper object model is good. Nothing forces you to base it on CatalogInfo though ;-p

CatalogInfo objects are instead known to the whole application, while you
can in theory change
the backend implementation everything works and plays against the
CatalogInfo sub-interface,
they define what we manage and what can be known about features, coverages
and the like.

What about using the CatalogInfo interface without registering this objects into the catalog ?. We can discuss integration if you see the whole thing in action ?

Yeah, we definitely need to avoid registering these objects with the catalog for a number of reasons:

  • catalog deals with data, not with services, not with security
  • putting these objects in such a prominent place would make them more than the default implementation,
    it would turn them into the only possibility (do you want to replace the entire catalog implementation to
    change the way you manage users?

Having a set of interfaces that matches the common case and that can be implemented against
a number of backends is good, the commonality will make it possible to implement a shared GUI
and set of REST services against all of them.
But it should not preclude the implementation of something entirely different, for which of course
we would not have a GUI and a REST api (they should disable themselves automatically in that case).

Short outline: I have 4 main classes
GeoserverUser implementing Spring UserDetails
GeoserverUserGroup which is new
GeoserverGrantedAuthority inherits Spring GrantedAuthority

Additionally, there is an Interface GeoserverSecuritySystem which implements
Spring UserDetailsService, a singleton is created by Spring injection.

To make this easily pluggable I’d suggest to follow the same approach as the authorization
subsytem: a fixed UserDetailsService implementation is registered in the spring context
that just plays the delegate.
This one looks into the spring context for the actual implementation, if one is found it
uses it, if none is found it instantiates one from code.
This way by default it would create the default implementation, but if someone drops
a custom implementation in the classpath it’s found and used.

I’m not sure what the rationale is for using property files in xml format
though?
As far as I know the only advantage over plain property files is that you
can store
multibyte chars in them?

I like the xml approach because of name spaces. As an example, I could combine the DDL xml file and the DML xml file into one xml file using 2 name spaces. If users dislike the xml property files, I could change that.

Warning, don’t force too much configuration down the user throat.
The DDL and DML should have defaults, leaving the possibility to customize
them only if the user really needs them.

Anyways, if you want to go property files I have nothing specific against
them,
though using XStream would have given you uniformity with how we already
manage configuration and rest resource encoding for the catalog, also with
interesting
bits on how to have the bits stored on disk relate to each other via ids,
whilst the
network format uses atom links in their place to ensure connectivity.

Not sure what your are meaning here, I did not invest much time how CatalogInfo works. I will start at DataStoreInfo to understand the architecture.

Here is an example, the release/workspaces/topp/states_shapefile/states/layer.xml
for the states layer, on disk. It links to the other catalog elements (styles, resource) by unique id:

states LayerInfoImpl--570ae188:124761b8d78:-7fc0 / VECTOR StyleInfoImpl--570ae188:124761b8d78:-7fe5 StyleInfoImpl--570ae188:124761b8d78:-7fe0 StyleInfoImpl--570ae188:124761b8d78:-7fee FeatureTypeInfoImpl--570ae188:124761b8d78:-7fc1 true 0 0

And here is the REST representation of the same, got from
http://localhost:8080/geoserver/rest/layers/states.xml

states / VECTOR population pophatch polygon states true 0 0

The same resources are linked by atom links, and the links can be followed: the xml
can be crawled by a web client jumping from one resource to the other, because the
links are explicit and direct instead of following the “same id” convention.

Did you already think about a REST api for managing users and permissions?
What would be the resources, their granularity and representation?

In a first step, I only want to handle the configuration using REST, but at the end of the day, I plan to offer the same possibilities for the GUI and the REST API.

Warning again, stuff that does not have a GUI it’s not really integrated in GeoServer.
People that prefer to mess with config files use MapServer, they are not our normal user base.
Unsurprisingly modules that require heavy xml based configuration are not popular.

Cheers
Andrea

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf