[Geoserver-devel] Advertised and Security filters with large JDBCConfig catalog

GeoServer GeoServer Developer
1

A client has identified a performance issue with very large JDBCConfig catalogs.

The AdvertisedCatalog and SecurityCatalog wrappers both add Filter objects (anonymous inner subclasses of InternalVolatileFilter) which JDBCConfig is unable to convert to SQL so it has to retrieve the objects in full, deserialize them, and then filter them all.

This prevents JDBCConfig from pushing paged listings and counting operations to the database. So for every page of the Layer Page or Layer Preview, it retreives and deserializes the full metadata for every single layer in the catalog, at least twice in order to simply show twenty items and a count of the total.

The Layer page can be resolved by not adding the filter if the request is not for a GetCapabilities request. It wasn’t doing anything for other requests.

Layer Preview does care about advertisability, and doesn’t use the filter system, but could be made much faster by doing paging and counting in the DB if it used filters none of the filters were unsupported by JDBCConfig.

The Advertised filter looks like it could be made to work with JDBCConfig although it’s potentially tricky due to the recursive definition of Advertisability (it depends on used resources or sublayers).

The Security filter is far more complicated though. However all it’s needed for is to hide inaccessible layers. Making layers that the user can’t read visible in those lists (the security wrapper would still prevent reading of the layers). Either an option to disable hiding these layers, or having JDBCConfig do it automatically would resolve this part of the problem.

Of the two, I favour an explicit option is it is security issue. It’s better to have sluggish performance for large JDBCConfig Catalogs that can be fixed if the user decides that the names of those layers being exposed is OK, rather than exposing potentially sensitive information by default.

Indicating that the layers are inaccessible, or even redacting information about them while still giving them a place in the list is also a possibility.

If anyone has any other ideas for getting around this problem I’d love to hear them.

–

Kevin Smith

Junior Software Engineer | Boundless

ksmith@anonymised.com

+1-778-785-7459

@boundlessgeo

2

There is a similar discussion on the user list “jdbcconfig - performance issue with 11K layers”.

···

Jody Garnett

On Fri, Mar 14, 2014 at 7:14 AM, Kevin Smith <ksmith@anonymised.com> wrote:

A client has identified a performance issue with very large JDBCConfig catalogs.

The AdvertisedCatalog and SecurityCatalog wrappers both add Filter objects (anonymous inner subclasses of InternalVolatileFilter) which JDBCConfig is unable to convert to SQL so it has to retrieve the objects in full, deserialize them, and then filter them all.

This prevents JDBCConfig from pushing paged listings and counting operations to the database. So for every page of the Layer Page or Layer Preview, it retreives and deserializes the full metadata for every single layer in the catalog, at least twice in order to simply show twenty items and a count of the total.

The Layer page can be resolved by not adding the filter if the request is not for a GetCapabilities request. It wasn’t doing anything for other requests.

Layer Preview does care about advertisability, and doesn’t use the filter system, but could be made much faster by doing paging and counting in the DB if it used filters none of the filters were unsupported by JDBCConfig.

The Advertised filter looks like it could be made to work with JDBCConfig although it’s potentially tricky due to the recursive definition of Advertisability (it depends on used resources or sublayers).

The Security filter is far more complicated though. However all it’s needed for is to hide inaccessible layers. Making layers that the user can’t read visible in those lists (the security wrapper would still prevent reading of the layers). Either an option to disable hiding these layers, or having JDBCConfig do it automatically would resolve this part of the problem.

Of the two, I favour an explicit option is it is security issue. It’s better to have sluggish performance for large JDBCConfig Catalogs that can be fixed if the user decides that the names of those layers being exposed is OK, rather than exposing potentially sensitive information by default.

Indicating that the layers are inaccessible, or even redacting information about them while still giving them a place in the list is also a possibility.

If anyone has any other ideas for getting around this problem I’d love to hear them.

–

Kevin Smith

Junior Software Engineer | Boundless

ksmith@anonymised.com

+1-778-785-7459

@boundlessgeo

Learn Graph Databases - Download FREE O’Reilly Book
“Graph Databases” is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

3

On Thu, Mar 13, 2014 at 9:14 PM, Kevin Smith <ksmith@anonymised.com>wrote:

A client has identified a performance issue with very large JDBCConfig
catalogs.

The AdvertisedCatalog and SecurityCatalog wrappers both add Filter objects
(anonymous inner subclasses of InternalVolatileFilter) which JDBCConfig is
unable to convert to SQL so it has to retrieve the objects in full,
deserialize them, and then filter them all.

This prevents JDBCConfig from pushing paged listings and counting
operations to the database. So for every page of the Layer Page or Layer
Preview, it retreives and deserializes the full metadata for every single
layer in the catalog, at least twice in order to simply show twenty items
and a count of the total.

Hi,
just to be sure, is the JDBCConfig code already doing fliter splitting
using the PostPreProcessFilterSplittingVisitor, to separate a pre-filter
that can be encoded, to a post filter that has to be run afterwards in
memory?

The Layer page can be resolved by not adding the filter if the request is
not for a GetCapabilities request. It wasn't doing anything for other
requests.

Layer Preview does care about advertisability, and doesn't use the filter
system, but could be made much faster by doing paging and counting in the
DB if it used filters none of the filters were unsupported by JDBCConfig.

The Advertised filter looks like it could be made to work with JDBCConfig
although it's potentially tricky due to the recursive definition of
Advertisability (it depends on used resources or sublayers).

I guess you could at least add some pre-filter for the non recursive
portion of it?

The Security filter is far more complicated though. However all it's
needed for is to hide inaccessible layers. Making layers that the user
can't read visible in those lists (the security wrapper would still prevent
reading of the layers). Either an option to disable hiding these layers, or
having JDBCConfig do it automatically would resolve this part of the
problem.

Hmm... the current ResourceAccessManager is designed to work against an in
memory catalog, it gives you access resource
by resource, in order to deal with a large amount of layer with security it
needs to be modified to return a Filter that will
locate, or at least pre-filter, all accessible objects of a given kind,
something like:

Filter getAccessibleCatalgoInfos(Class catalogInfoType);

which would for example be called like:
ram.getAccessibleCatalgoInfos(DataStoreInfo.class); --> filter that
restricts the stores we can see
ram.getAccessibleCatalgoInfos(LayerInfo.class); --> filter that restricts
the layers we can see

and then the existing methods would be used either as a secondary filter
(maybe some parts of the access control
cannot be turned into a filter) and/or to apply access lmitations (e.g.
yes, you can see layer x, but only a subset of its attributes).
This will break existing implementations of ResourceAccessManager, so it
requires a GSIP.

Cheers
Andrea

--

Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

4

On 14 March 2014 08:49, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Thu, Mar 13, 2014 at 9:14 PM, Kevin Smith <ksmith@anonymised.com>wrote:

A client has identified a performance issue with very large JDBCConfig
catalogs.

The AdvertisedCatalog and SecurityCatalog wrappers both add Filter
objects (anonymous inner subclasses of InternalVolatileFilter) which
JDBCConfig is unable to convert to SQL so it has to retrieve the objects in
full, deserialize them, and then filter them all.

This prevents JDBCConfig from pushing paged listings and counting
operations to the database. So for every page of the Layer Page or Layer
Preview, it retreives and deserializes the full metadata for every single
layer in the catalog, at least twice in order to simply show twenty items
and a count of the total.

Hi,
just to be sure, is the JDBCConfig code already doing fliter splitting
using the PostPreProcessFilterSplittingVisitor, to separate a pre-filter
that can be encoded, to a post filter that has to be run afterwards in
memory?

It's using CapabilitiesFilterSplitter to do the same thing.

The Layer page can be resolved by not adding the filter if the request is
not for a GetCapabilities request. It wasn't doing anything for other
requests.

Layer Preview does care about advertisability, and doesn't use the filter
system, but could be made much faster by doing paging and counting in the
DB if it used filters none of the filters were unsupported by JDBCConfig.

The Advertised filter looks like it could be made to work with JDBCConfig
although it's potentially tricky due to the recursive definition of
Advertisability (it depends on used resources or sublayers).

I guess you could at least add some pre-filter for the non recursive
portion of it?

Yes, although the presence of post-filters at all is the big problem.
Moving some of the logic to pre will help a little, but to get the big
performance payoff, at least for cases like the Layer Preview page, it has
to all be pre.

It appeared that AdvertisedCatalog assumes layer groups could have
sub-layergroups while LayerPreviewProvider assumes they only contain
layers. If it does need to recurse through nested groups, a CTE query
should be able to handle it, although not all DBs support that so there
would have to be a check for SQL dialect and a fall back to a post filter
if CTE is not supported.

The Security filter is far more complicated though. However all it's
needed for is to hide inaccessible layers. Making layers that the user
can't read visible in those lists (the security wrapper would still prevent
reading of the layers). Either an option to disable hiding these layers, or
having JDBCConfig do it automatically would resolve this part of the
problem.

Hmm... the current ResourceAccessManager is designed to work against an in
memory catalog, it gives you access resource
by resource, in order to deal with a large amount of layer with security
it needs to be modified to return a Filter that will
locate, or at least pre-filter, all accessible objects of a given kind,
something like:

Filter getAccessibleCatalgoInfos(Class catalogInfoType);

which would for example be called like:
ram.getAccessibleCatalgoInfos(DataStoreInfo.class); --> filter that
restricts the stores we can see
ram.getAccessibleCatalgoInfos(LayerInfo.class); --> filter that restricts
the layers we can see

and then the existing methods would be used either as a secondary filter
(maybe some parts of the access control
cannot be turned into a filter) and/or to apply access lmitations (e.g.
yes, you can see layer x, but only a subset of its attributes).
This will break existing implementations of ResourceAccessManager, so it
requires a GSIP.

If we leave anything as a post-filter it will disable database side paging
and counting which are what I'm trying to get working. Wrappers are fine
though. I tried to follow the logic of how the current filter is built,
but there seem to be too many places where new behaviour can be plugged in
with extension points and wrappers for it to be practical to re-implement
in SQL. I'm not too familiar with the security system though. I didn't go
much beyond the buildWrapperPolicy methods on SecureCatalogImpl while
looking into this.

--

Kevin Smith

Junior Software Engineer | Boundless

ksmith@anonymised.com

+1-778-785-7459

@boundlessgeo <https://twitter.com/boundlessgeo&gt;

5

On Fri, Mar 14, 2014 at 6:59 PM, Kevin Smith <ksmith@anonymised.com>wrote:

Yes, although the presence of post-filters at all is the big problem.
Moving some of the logic to pre will help a little, but to get the big
performance payoff, at least for cases like the Layer Preview page, it has
to all be pre.

It appeared that AdvertisedCatalog assumes layer groups could have
sub-layergroups while LayerPreviewProvider assumes they only contain
layers.

Ah, the latter seems an oversight of when nested layer groups where
implemented, it should work like AdvertisedCatalog instead.

If it does need to recurse through nested groups, a CTE query should be
able to handle it, although not all DBs support that so there would have to
be a check for SQL dialect and a fall back to a post filter if CTE is not
supported.

Right. Alternatively, implement some sort of derived property that you have
in the DB, and that is getting updated recursively when a nested attribute
changes

The Security filter is far more complicated though. However all it's
needed for is to hide inaccessible layers. Making layers that the user
can't read visible in those lists (the security wrapper would still prevent
reading of the layers). Either an option to disable hiding these layers, or
having JDBCConfig do it automatically would resolve this part of the
problem.

Hmm... the current ResourceAccessManager is designed to work against an
in memory catalog, it gives you access resource
by resource, in order to deal with a large amount of layer with security
it needs to be modified to return a Filter that will
locate, or at least pre-filter, all accessible objects of a given kind,
something like:

Filter getAccessibleCatalgoInfos(Class catalogInfoType);

which would for example be called like:
ram.getAccessibleCatalgoInfos(DataStoreInfo.class); --> filter that
restricts the stores we can see
ram.getAccessibleCatalgoInfos(LayerInfo.class); --> filter that restricts
the layers we can see

and then the existing methods would be used either as a secondary filter
(maybe some parts of the access control
cannot be turned into a filter) and/or to apply access lmitations (e.g.
yes, you can see layer x, but only a subset of its attributes).
This will break existing implementations of ResourceAccessManager, so it
requires a GSIP.

If we leave anything as a post-filter it will disable database side paging
and counting which are what I'm trying to get working. Wrappers are fine
though. I tried to follow the logic of how the current filter is built,
but there seem to be too many places where new behaviour can be plugged in
with extension points and wrappers for it to be practical to re-implement
in SQL. I'm not too familiar with the security system though. I didn't go
much beyond the buildWrapperPolicy methods on SecureCatalogImpl while
looking into this.

I cannot guarantee a security implementation can do all its work without
doing post filtering (as noted above, it might have to drill down in the
object structure to check some attribute in referenced entities). But we
could have an interface that tells if you if post filtering is going to be
needed, or not, and lets you optimize the first case.

Cheers
Andrea

--

Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

6

On 14 March 2014 11:06, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Fri, Mar 14, 2014 at 6:59 PM, Kevin Smith <ksmith@anonymised.com>wrote:

It appeared that AdvertisedCatalog assumes layer groups could have
sub-layergroups while LayerPreviewProvider assumes they only contain
layers.

Ah, the latter seems an oversight of when nested layer groups where
implemented, it should work like AdvertisedCatalog instead.

Yeah, getting them to share an implementation would be a side benefit.

If it does need to recurse through nested groups, a CTE query should be
able to handle it, although not all DBs support that so there would have to
be a check for SQL dialect and a fall back to a post filter if CTE is not
supported.

Right. Alternatively, implement some sort of derived property that you
have in the DB, and that is getting updated recursively when a nested
attribute changes

Yes, although I'd prefer not to mess with the JDBCConfig schema if I don't
have to. I suppose it would have a performance benefit on reading over
computing it during the query.

I cannot guarantee a security implementation can do all its work without
doing post filtering (as noted above, it might have to drill down in the
object structure to check some attribute in referenced entities). But we
could have an interface that tells if you if post filtering is going to be
needed, or not, and lets you optimize the first case.

Allowing the admin to specify that it's OK to reveal the existence of
secure layers would be a simple and comprehensive way to signal that the
filter isn't needed. Even if we implemented partial pre-filtering, it
would still be useful to improve performance when pre-filtering is not
possible and metadata security is not required. Its certainly a first step
we could add comparatively easily. Maybe three options: Always Hide, Hide
if Prefilterable, Never Hide. Hide gives metadata security, Hide if
Prefilterable gives the best performance, Never Hide may improve on Always
Hide for performance depending on the proportion of layers that would be
pre-filtered and is more consistent in behaviour than Hide if Prefilterable.

--

Kevin Smith

Junior Software Engineer | Boundless

ksmith@anonymised.com

+1-778-785-7459

@boundlessgeo <https://twitter.com/boundlessgeo&gt;

7

On Fri, Mar 14, 2014 at 11:27 PM, Kevin Smith <ksmith@anonymised.com>wrote:

Allowing the admin to specify that it's OK to reveal the existence of
secure layers would be a simple and comprehensive way to signal that the
filter isn't needed. Even if we implemented partial pre-filtering, it
would still be useful to improve performance when pre-filtering is not
possible and metadata security is not required. Its certainly a first step
we could add comparatively easily. Maybe three options: Always Hide, Hide
if Prefilterable, Never Hide. Hide gives metadata security, Hide if
Prefilterable gives the best performance, Never Hide may improve on Always
Hide for performance depending on the proportion of layers that would be
pre-filtered and is more consistent in behaviour than Hide if Prefilterable.

That makes sense from the point of view of JDBC config, but I don't think
it makes sense for the administrator to see such an implementation detail
(I assume you want to extend the existing catalog policies
hide/mixed/challenge, or make it otherwise configurable in the GUI?)

I don't see anything wrong in ResourceAccessManager returning a filter to
prefilter and a flag stating if pre-filtering is all there is to it, or if
some layer by layer post filtering is needed, explaining in the interface
the consequences of
a layer by layer post filtering in terms of performance. In the end, to
achieve any form of pre-filtering, you'll have to change that interface
anyways. Someone must put some time thinking on how to best change it btw,
so far I've just took the time to answer your mails, it would be best to
spend some hours looking at how the interface is used and implemented in
the current system to see if other improvements are needed JDBC config wise.

Cheers
Andrea

--

Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

8

Ok, I feel a bit silly for missing that Catalog Mode option already existed. Knowing that, my idea boils don to having SecureCatalogImpl not add the filter when in challenge mode, rather than adding a filter that will be applied to each file, only to find that it’s challenge mode so all the files should be allowed through.

···

On 16 March 2014 00:56, Andrea Aime <andrea.aime@anonymised.com> wrote:

–

Kevin Smith

Junior Software Engineer | Boundless

ksmith@anonymised.com

+1-778-785-7459

@boundlessgeo

On Fri, Mar 14, 2014 at 11:27 PM, Kevin Smith <ksmith@anonymised.com> wrote:

That makes sense from the point of view of JDBC config, but I don’t think it makes sense for the administrator to see such an implementation detail (I assume you want to extend the existing catalog policies hide/mixed/challenge, or make it otherwise configurable in the GUI?)

I don’t see anything wrong in ResourceAccessManager returning a filter to prefilter and a flag stating if pre-filtering is all there is to it, or if some layer by layer post filtering is needed, explaining in the interface the consequences of
a layer by layer post filtering in terms of performance. In the end, to achieve any form of pre-filtering, you’ll have to change that interface anyways. Someone must put some time thinking on how to best change it btw, so far I’ve just took the time to answer your mails, it would be best to spend some hours looking at how the interface is used and implemented in the current system to see if other improvements are needed JDBC config wise.

Cheers

Andrea

–

==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

Allowing the admin to specify that it’s OK to reveal the existence of secure layers would be a simple and comprehensive way to signal that the filter isn’t needed. Even if we implemented partial pre-filtering, it would still be useful to improve performance when pre-filtering is not possible and metadata security is not required. Its certainly a first step we could add comparatively easily. Maybe three options: Always Hide, Hide if Prefilterable, Never Hide. Hide gives metadata security, Hide if Prefilterable gives the best performance, Never Hide may improve on Always Hide for performance depending on the proportion of layers that would be pre-filtered and is more consistent in behaviour than Hide if Prefilterable.

9

I’ve done some more digging on this and just turning the filter on and off in SecureCatalogImpl isn’t going to work. My current thought for handling the security side is to extend the ResourceAccessManager interface with a method that returns an org.opengis.filter.Filter which filters out any catalog object that should be hidden. If it’s not a “well known” filter, then we’re back where we started, but if the ResourceAccessManager can produce a well known filter, then JDBCConfig can encode it and we get the performance improvement.

Filter getFilter(Authentication user, Class<? extends CatalogInfo> clazz)

DataAccessManagerAdapter could return an include all filter when in challenge mode, and a filter akin to what SecureCatalogImpl builds otherwise.

CatalogFilterAccessManager would require adding a similar Filter generating method to the CatalogFilter interface, and then would combine the Filters produced by its CatalogFilters together and return that.

It converts as much as possible into well known filters and keeps the generation of those filters close to the code they need to mimic. If the catalog is in challenge mode, and no unusual CatalogFilter implementations are in use, it should allow the entire security filter to be well known and so allow for database side counting and paging, and so much better performance for those queries on large catalogs.

···

On 18 March 2014 17:22, Kevin Smith <ksmith@anonymised.com> wrote:

Ok, I feel a bit silly for missing that Catalog Mode option already existed. Knowing that, my idea boils don to having SecureCatalogImpl not add the filter when in challenge mode, rather than adding a filter that will be applied to each file, only to find that it’s challenge mode so all the files should be allowed through.

–

Kevin Smith

Junior Software Engineer | Boundless

ksmith@anonymised.com

+1-778-785-7459

@boundlessgeo

On 16 March 2014 00:56, Andrea Aime <andrea.aime@anonymised.com1268…> wrote:

–

Kevin Smith

Junior Software Engineer | Boundless

ksmith@anonymised.com

+1-778-785-7459

@boundlessgeo

On Fri, Mar 14, 2014 at 11:27 PM, Kevin Smith <ksmith@anonymised.com> wrote:

That makes sense from the point of view of JDBC config, but I don’t think it makes sense for the administrator to see such an implementation detail (I assume you want to extend the existing catalog policies hide/mixed/challenge, or make it otherwise configurable in the GUI?)

I don’t see anything wrong in ResourceAccessManager returning a filter to prefilter and a flag stating if pre-filtering is all there is to it, or if some layer by layer post filtering is needed, explaining in the interface the consequences of
a layer by layer post filtering in terms of performance. In the end, to achieve any form of pre-filtering, you’ll have to change that interface anyways. Someone must put some time thinking on how to best change it btw, so far I’ve just took the time to answer your mails, it would be best to spend some hours looking at how the interface is used and implemented in the current system to see if other improvements are needed JDBC config wise.

Cheers

Andrea

–

==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

Allowing the admin to specify that it’s OK to reveal the existence of secure layers would be a simple and comprehensive way to signal that the filter isn’t needed. Even if we implemented partial pre-filtering, it would still be useful to improve performance when pre-filtering is not possible and metadata security is not required. Its certainly a first step we could add comparatively easily. Maybe three options: Always Hide, Hide if Prefilterable, Never Hide. Hide gives metadata security, Hide if Prefilterable gives the best performance, Never Hide may improve on Always Hide for performance depending on the proportion of layers that would be pre-filtered and is more consistent in behaviour than Hide if Prefilterable.

10

On Thu, Apr 10, 2014 at 10:44 PM, Kevin Smith <ksmith@anonymised.com>wrote:

I've done some more digging on this and just turning the filter on and off
in SecureCatalogImpl isn't going to work. My current thought for handling
the security side is to extend the ResourceAccessManager interface with a
method that returns an org.opengis.filter.Filter which filters out any
catalog object that should be hidden. If it's not a "well known" filter,
then we're back where we started, but if the ResourceAccessManager can
produce a well known filter, then JDBCConfig can encode it and we get the
performance improvement.

Filter getFilter(Authentication user, Class<? extends CatalogInfo> clazz)

DataAccessManagerAdapter could return an include all filter when in
challenge mode, and a filter akin to what SecureCatalogImpl builds
otherwise.

CatalogFilterAccessManager would require adding a similar Filter
generating method to the CatalogFilter interface, and then would combine
the Filters produced by its CatalogFilters together and return that.

It converts as much as possible into well known filters and keeps the
generation of those filters close to the code they need to mimic. If the
catalog is in challenge mode, and no unusual CatalogFilter implementations
are in use, it should allow the entire security filter to be well known and
so allow for database side counting and paging, and so much better
performance for those queries on large catalogs.

+1

Since this is a interface change for a published extension point that other
projects use (GeoShield, GeoFence just to cite the public ones I'm aware
of), you'll have to make a (quick) proposal

Cheers
Andrea

--

Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

11

FWIW, +1 here too.
As Andrea mentioned it’ll require a small proposal but it seems like just to comply with the little bureaucracy

···

On Thu, Apr 10, 2014 at 5:53 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test & Deployment
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

Gabriel Roldán

Software Developer | Boundless

groldan@anonymised.com

@gabrielroldan

On Thu, Apr 10, 2014 at 10:44 PM, Kevin Smith <ksmith@anonymised.com…> wrote:

I’ve done some more digging on this and just turning the filter on and off in SecureCatalogImpl isn’t going to work. My current thought for handling the security side is to extend the ResourceAccessManager interface with a method that returns an org.opengis.filter.Filter which filters out any catalog object that should be hidden. If it’s not a “well known” filter, then we’re back where we started, but if the ResourceAccessManager can produce a well known filter, then JDBCConfig can encode it and we get the performance improvement.

Filter getFilter(Authentication user, Class<? extends CatalogInfo> clazz)

DataAccessManagerAdapter could return an include all filter when in challenge mode, and a filter akin to what SecureCatalogImpl builds otherwise.

CatalogFilterAccessManager would require adding a similar Filter generating method to the CatalogFilter interface, and then would combine the Filters produced by its CatalogFilters together and return that.

It converts as much as possible into well known filters and keeps the generation of those filters close to the code they need to mimic. If the catalog is in challenge mode, and no unusual CatalogFilter implementations are in use, it should allow the entire security filter to be well known and so allow for database side counting and paging, and so much better performance for those queries on large catalogs.

+1

Since this is a interface change for a published extension point that other projects use (GeoShield, GeoFence just to cite the public ones I’m aware of), you’ll have to make a (quick) proposal

Cheers

Andrea

–

==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

12

On Thu, Apr 10, 2014 at 11:54 PM, Gabriel Roldan
<groldan@anonymised.com>wrote:

FWIW, +1 here too.
As Andrea mentioned it'll require a small proposal but it seems like just
to comply with the little bureaucracy

I'd like to actually point out the change to the wider community, having a
proposal helps as it has a
page that we can link to.

As said, there are various implementation of ResourceAccessManager in the
wild (GeoShield and GeoFence
are just two example, I know of other ones used to integrate GeoServer with
in house auth systems),
so we'd better ask around before breaking it.

Cheers
Andrea

--

Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

13

Kevin has set up the proposal here:

http://geoserver.org/display/GEOS/GSIP+113±+ResourceAccessManager+to+Build+Security+Filter

···

Jody Garnett

On Fri, Apr 11, 2014 at 4:47 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test & Deployment
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Thu, Apr 10, 2014 at 11:54 PM, Gabriel Roldan <groldan@anonymised.com3839…> wrote:

FWIW, +1 here too.
As Andrea mentioned it’ll require a small proposal but it seems like just to comply with the little bureaucracy

I’d like to actually point out the change to the wider community, having a proposal helps as it has a
page that we can link to.

As said, there are various implementation of ResourceAccessManager in the wild (GeoShield and GeoFence
are just two example, I know of other ones used to integrate GeoServer with in house auth systems),
so we’d better ask around before breaking it.

Cheers

Andrea

–

==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it