[Geoserver-devel] Non advertised layers, a bit of design discussion

GeoServer GeoServer Developer
1

Hi,
coming back on the "non advertised" layers work, which I should start shortly,
I was thinking about how to implement it.

The last discussion on the topic concluded that it would be nice to have
a generic catalog wrapper that can filter out stuff based on certain context
conditions, the advertising filter being driver by the current request
(getcapabilities)
and a layer flag (advertised or not).

Now, we already have two levels of catalog filtering:
- the secured catalog, which in turns talks to a ResourceAccessManager
- the per workspace filter, which is a ResourceAccessManager wrapper

So I was wondering, why not continue on this road?
Writing a catalog wrapper is long and tedius and will result in
significant maintenance
work if we have to add new entities to the catalog.

There are a few options I have in mind:

1) the "composite" plan: have people register multiple ResourceAccessManager
   objects in the app context and combine them into a composite.
   The tricky part here is to combine the access limits.
   Filters can be and-ed, the attribute lists can be intersected, so
can cookie cutter
   geometries, but the catalog access mode... uh, not sure what to do about it.
   HIDE is stricter than MIXED which is stricter than CHALLENGE?

2) the "filter" plan: we keep just one resource access manager
wrapper, implemented
    pretty much like LocalWorkspaceResourceAccessManager, which delegates to
    a number of CatalogFilter objects looked up in the spring context.

CatalogFilter {
   boolean hideLayer(LayerInfo);
   boolean hideWorkspace(WorkspaceInfo);
   boolean hdeResource(ResourceInfo);
}

   The first that down-votes a resource makes it hidden

I'm leaning towards the second approach, since the first would break the "give
me a ResourceAccessManager from the spring context or I'll instantiate the
default one" approach that we have today, and also because the second is
simpler.

Opinions?

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

2

On Wed, Sep 28, 2011 at 9:22 AM, Andrea Aime <andrea.aime@anonymised.com.1268…> wrote:

  1. the “filter” plan: we keep just one resource access manager
    wrapper, implemented
    pretty much like LocalWorkspaceResourceAccessManager, which delegates to
    a number of CatalogFilter objects looked up in the spring context.

CatalogFilter {
boolean hideLayer(LayerInfo);
boolean hideWorkspace(WorkspaceInfo);
boolean hdeResource(ResourceInfo);
}

The first that down-votes a resource makes it hidden

Hey,
whipped up a quick patch following this approach, seems promising.
The patch is attached. Mind, it’s not done, does not have copyright
headers, no tests, but it seems to work in my limited interactive testing.

The 10000 feet view of the patch:

  • follows the filter approach
  • there are two filters, one to hide the layers in other workspaces, one to
    hide the non advertised layers/resources from the caps requests
  • the patch is for trunk, it has the new field but also the backwards compatibility
    using the metadata key, the 2.1.x patch will only have the metadata lookup instead
  • the flag had to be added both to layer and resource, pretty much like the enabled
    flag, since wms caps uses layers, but wfs and wcs ask for the resources directly.
    Just like the resource enabled flag there is code to keep the two classes in synch

Feedback welcomed

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

(attachments)

hide.patch (25.6 KB)

3

Nice work Andrea! Very clean. I like the second approach as well… much more direct and simplifies the virtual servies / local workspace stuff. +1.

On Wed, Sep 28, 2011 at 6:30 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Wed, Sep 28, 2011 at 9:22 AM, Andrea Aime <andrea.aime@anonymised.com.> wrote:

  1. the “filter” plan: we keep just one resource access manager
    wrapper, implemented
    pretty much like LocalWorkspaceResourceAccessManager, which delegates to
    a number of CatalogFilter objects looked up in the spring context.

CatalogFilter {
boolean hideLayer(LayerInfo);
boolean hideWorkspace(WorkspaceInfo);
boolean hdeResource(ResourceInfo);
}

The first that down-votes a resource makes it hidden

Hey,
whipped up a quick patch following this approach, seems promising.
The patch is attached. Mind, it’s not done, does not have copyright
headers, no tests, but it seems to work in my limited interactive testing.

The 10000 feet view of the patch:

  • follows the filter approach
  • there are two filters, one to hide the layers in other workspaces, one to
    hide the non advertised layers/resources from the caps requests
  • the patch is for trunk, it has the new field but also the backwards compatibility
    using the metadata key, the 2.1.x patch will only have the metadata lookup instead
  • the flag had to be added both to layer and resource, pretty much like the enabled
    flag, since wms caps uses layers, but wfs and wcs ask for the resources directly.
    Just like the resource enabled flag there is code to keep the two classes in synch

Feedback welcomed

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2dcopy1

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

4

Actually one bit of feedback, and it relates to the other thread on virtual services. I was thinking about how to implement layer groups and styles local to workspace and thought of an interface like this that could be used to filter by workspace the same way it does for resources. Anyways that is a topic of another discussion but it might be nice to make CatalogFilter an abstract class… in case that is the route we want to go. $0.02

On Wed, Sep 28, 2011 at 7:39 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Nice work Andrea! Very clean. I like the second approach as well… much more direct and simplifies the virtual servies / local workspace stuff. +1.

On Wed, Sep 28, 2011 at 6:30 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Wed, Sep 28, 2011 at 9:22 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

  1. the “filter” plan: we keep just one resource access manager
    wrapper, implemented
    pretty much like LocalWorkspaceResourceAccessManager, which delegates to
    a number of CatalogFilter objects looked up in the spring context.

CatalogFilter {
boolean hideLayer(LayerInfo);
boolean hideWorkspace(WorkspaceInfo);
boolean hdeResource(ResourceInfo);
}

The first that down-votes a resource makes it hidden

Hey,
whipped up a quick patch following this approach, seems promising.
The patch is attached. Mind, it’s not done, does not have copyright
headers, no tests, but it seems to work in my limited interactive testing.

The 10000 feet view of the patch:

  • follows the filter approach
  • there are two filters, one to hide the layers in other workspaces, one to
    hide the non advertised layers/resources from the caps requests
  • the patch is for trunk, it has the new field but also the backwards compatibility
    using the metadata key, the 2.1.x patch will only have the metadata lookup instead
  • the flag had to be added both to layer and resource, pretty much like the enabled
    flag, since wms caps uses layers, but wfs and wcs ask for the resources directly.
    Just like the resource enabled flag there is code to keep the two classes in synch

Feedback welcomed

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2dcopy1

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

5

It seems we could have “interface CatalogFilter” and “abstract class AbstractCatalogFilter implements CatalogFilter” to provide a convenient base class while still having the flexibility of an interface. Or am I misinterpreting your suggestion?

–
David Winslow
OpenGeo - http://opengeo.org/

On Wed, Sep 28, 2011 at 9:43 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Actually one bit of feedback, and it relates to the other thread on virtual services. I was thinking about how to implement layer groups and styles local to workspace and thought of an interface like this that could be used to filter by workspace the same way it does for resources. Anyways that is a topic of another discussion but it might be nice to make CatalogFilter an abstract class… in case that is the route we want to go. $0.02

On Wed, Sep 28, 2011 at 7:39 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Nice work Andrea! Very clean. I like the second approach as well… much more direct and simplifies the virtual servies / local workspace stuff. +1.

On Wed, Sep 28, 2011 at 6:30 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Wed, Sep 28, 2011 at 9:22 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

  1. the “filter” plan: we keep just one resource access manager
    wrapper, implemented
    pretty much like LocalWorkspaceResourceAccessManager, which delegates to
    a number of CatalogFilter objects looked up in the spring context.

CatalogFilter {
boolean hideLayer(LayerInfo);
boolean hideWorkspace(WorkspaceInfo);
boolean hdeResource(ResourceInfo);
}

The first that down-votes a resource makes it hidden

Hey,
whipped up a quick patch following this approach, seems promising.
The patch is attached. Mind, it’s not done, does not have copyright
headers, no tests, but it seems to work in my limited interactive testing.

The 10000 feet view of the patch:

  • follows the filter approach
  • there are two filters, one to hide the layers in other workspaces, one to
    hide the non advertised layers/resources from the caps requests
  • the patch is for trunk, it has the new field but also the backwards compatibility
    using the metadata key, the 2.1.x patch will only have the metadata lookup instead
  • the flag had to be added both to layer and resource, pretty much like the enabled
    flag, since wms caps uses layers, but wfs and wcs ask for the resources directly.
    Just like the resource enabled flag there is code to keep the two classes in synch

Feedback welcomed

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2dcopy1

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2dcopy1

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

6

On Wed, Sep 28, 2011 at 3:43 PM, Justin Deoliveira <jdeolive@anonymised.com.1501…> wrote:

Actually one bit of feedback, and it relates to the other thread on virtual services. I was thinking about how to implement layer groups and styles local to workspace and thought of an interface like this that could be used to filter by workspace the same way it does for resources. Anyways that is a topic of another discussion but it might be nice to make CatalogFilter an abstract class… in case that is the route we want to go. $0.02

I can go for that, there is just a hiccup: the security subsystem has no notion of the layer groups,
because no request ever gets down to the catalog with them, they are split into their components
at the kvp parser level.
So whoever wants to go there will have to change teh ResourceAccessManager interface to
include layer groups, change the secure catalog impl accordingly, and yes, then implement
the code in the catalog filters.

Btw, I’m not sure about the CatalogFilter name… it’s not like it really filters everything in the
catalog.
Though in fact if we make it an abstract class we’re free to add new types of filters afterwards

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

7

On Wed, Sep 28, 2011 at 3:39 PM, Justin Deoliveira <jdeolive@anonymised.com.1501…> wrote:

Nice work Andrea! Very clean. I like the second approach as well… much more direct and simplifies the virtual servies / local workspace stuff. +1.

Cool, I’ll put togheter the rest of the work then (tests, javadocs and the like)

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

8

On Wed, Sep 28, 2011 at 7:58 AM, David Winslow <dwinslow@anonymised.com> wrote:

It seems we could have “interface CatalogFilter” and “abstract class AbstractCatalogFilter implements CatalogFilter” to provide a convenient base class while still having the flexibility of an interface. Or am I misinterpreting your suggestion?

Nope, straight abstract class or abstract class + interface is what I meant. And indeed the latter is more flexible for classes that already extend from another base class. Just as long as its clear that you should extend the base class if possible to make maintaining the interface easier.

–
David Winslow
OpenGeo - http://opengeo.org/

On Wed, Sep 28, 2011 at 9:43 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Actually one bit of feedback, and it relates to the other thread on virtual services. I was thinking about how to implement layer groups and styles local to workspace and thought of an interface like this that could be used to filter by workspace the same way it does for resources. Anyways that is a topic of another discussion but it might be nice to make CatalogFilter an abstract class… in case that is the route we want to go. $0.02

On Wed, Sep 28, 2011 at 7:39 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Nice work Andrea! Very clean. I like the second approach as well… much more direct and simplifies the virtual servies / local workspace stuff. +1.

On Wed, Sep 28, 2011 at 6:30 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Wed, Sep 28, 2011 at 9:22 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

  1. the “filter” plan: we keep just one resource access manager
    wrapper, implemented
    pretty much like LocalWorkspaceResourceAccessManager, which delegates to
    a number of CatalogFilter objects looked up in the spring context.

CatalogFilter {
boolean hideLayer(LayerInfo);
boolean hideWorkspace(WorkspaceInfo);
boolean hdeResource(ResourceInfo);
}

The first that down-votes a resource makes it hidden

Hey,
whipped up a quick patch following this approach, seems promising.
The patch is attached. Mind, it’s not done, does not have copyright
headers, no tests, but it seems to work in my limited interactive testing.

The 10000 feet view of the patch:

  • follows the filter approach
  • there are two filters, one to hide the layers in other workspaces, one to
    hide the non advertised layers/resources from the caps requests
  • the patch is for trunk, it has the new field but also the backwards compatibility
    using the metadata key, the 2.1.x patch will only have the metadata lookup instead
  • the flag had to be added both to layer and resource, pretty much like the enabled
    flag, since wms caps uses layers, but wfs and wcs ask for the resources directly.
    Just like the resource enabled flag there is code to keep the two classes in synch

Feedback welcomed

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2dcopy1

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2dcopy1

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

9

On Wed, Sep 28, 2011 at 8:13 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Wed, Sep 28, 2011 at 3:43 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Actually one bit of feedback, and it relates to the other thread on virtual services. I was thinking about how to implement layer groups and styles local to workspace and thought of an interface like this that could be used to filter by workspace the same way it does for resources. Anyways that is a topic of another discussion but it might be nice to make CatalogFilter an abstract class… in case that is the route we want to go. $0.02

I can go for that, there is just a hiccup: the security subsystem has no notion of the layer groups,
because no request ever gets down to the catalog with them, they are split into their components
at the kvp parser level.

So whoever wants to go there will have to change teh ResourceAccessManager interface to
include layer groups, change the secure catalog impl accordingly, and yes, then implement
the code in the catalog filters.

Yeah this was generally what I was thinking. Again I am not sure if its the route to go it was just my current train of thought.

Btw, I’m not sure about the CatalogFilter name… it’s not like it really filters everything in the
catalog.

Hmmm… I think filter makes sense… I could also see this interface being extended to other use cases. Like perhaps used in an extended catalog query api… with maybe some subclasses of the interface that things like the hibernate catalog can translate into native filters. Just spitballing. But open to some other name too.

Though in fact if we make it an abstract class we’re free to add new types of filters afterwards

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.