[Geoserver-devel] geofence: invalidating rules cache

GeoServer GeoServer Developer
1

Hello Nuno,

Another change I have been asked to make in geofence, is that rules in the cache would automatically invalidated when they are changed. I have looked at the code, and I have an idea how to do it, and was wondering if you agree with the approach:

1. write a RuleFilter.matches(Rule rule) method (I think currently rule filters are only used to translate into queries, and there is no way to match a filter with a single rule)

2. write a CachedRuleReader.invalidate(Rule rule) which loops through the whole cache, and invalidates all rule filters that match the rule.

3. write a CachedRuleAdminServiceImpl, that delegates to the normal RuleAdminServiceImpl but calls the above invalidate upon change of an existing rule.

Kind Regards

Niels

2

Hi Niels,

I’m nor very familiar with that portion of the code … but here are my thoughts.

Well the workflow makes sense: if a rule is deleted or modified that rule is invalidated from the cache.

The rule matches and invalidate should probably be done by the rule ID and not the rule itself, so I guess instead of:

CachedRuleReader.invalidate(Rule rule)

we should probably have something like:

CachedRuleReader.invalidate(String id)

Regards,

Nuno Oliveira

···

On 02-06-2017 12:46, Niels Charlier wrote:

Hello Nuno,

Another change I have been asked to make in geofence, is that rules in the cache would automatically invalidated when they are changed. I have looked at the code, and I have an idea how to do it, and was wondering if you agree with the approach:

  1. write a RuleFilter.matches(Rule rule) method (I think currently rule filters are only used to translate into queries, and there is no way to match a filter with a single rule)

  2. write a CachedRuleReader.invalidate(Rule rule) which loops through the whole cache, and invalidates all rule filters that match the rule.

  3. write a CachedRuleAdminServiceImpl, that delegates to the normal RuleAdminServiceImpl but calls the above invalidate upon change of an existing rule.

Kind Regards

Niels

-- 
==
GeoServer Professional Services from the experts! 
Visit [http://goo.gl/it488V](http://goo.gl/it488V) for more information.
==
Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy

phone: +39 0584 962313
fax:   +39 0584 1660272
mob:   +39  333 8128928

[http://www.geo-solutions.it](http://www.geo-solutions.it)
[http://twitter.com/geosolutions_it](http://twitter.com/geosolutions_it)

-------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono
da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate
nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e
-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo
anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.
 
The information in this message and/or attachments, is intended solely for the attention and use of
the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree
June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying,
distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender does not give any warranty or accept liability as the content,
accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which
arise as a result of e-mail transmission, viruses, etc.
3

In that case it will still need to pick up the whole rule, because otherwise it can’t match the rule filters with the rule.

But RuleAdminServiceImpl has the entire rule already upon change, so it seems like that would introduce an unnecessary step

···

On 05-06-17 11:11, Nuno Oliveira wrote:

Hi Niels,

I’m nor very familiar with that portion of the code … but here are my thoughts.

Well the workflow makes sense: if a rule is deleted or modified that rule is invalidated from the cache.

The rule matches and invalidate should probably be done by the rule ID and not the rule itself, so I guess instead of:

CachedRuleReader.invalidate(Rule rule)

we should probably have something like:

CachedRuleReader.invalidate(String id)

Regards,

Nuno Oliveira

On 02-06-2017 12:46, Niels Charlier wrote:

Hello Nuno,

Another change I have been asked to make in geofence, is that rules in the cache would automatically invalidated when they are changed. I have looked at the code, and I have an idea how to do it, and was wondering if you agree with the approach:

  1. write a RuleFilter.matches(Rule rule) method (I think currently rule filters are only used to translate into queries, and there is no way to match a filter with a single rule)

  2. write a CachedRuleReader.invalidate(Rule rule) which loops through the whole cache, and invalidates all rule filters that match the rule.

  3. write a CachedRuleAdminServiceImpl, that delegates to the normal RuleAdminServiceImpl but calls the above invalidate upon change of an existing rule.

Kind Regards

Niels

-- 
==
GeoServer Professional Services from the experts! 
Visit [http://goo.gl/it488V](http://goo.gl/it488V) for more information.
==
Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy

phone: +39 0584 962313
fax:   +39 0584 1660272
mob:   +39  333 8128928

[http://www.geo-solutions.it](http://www.geo-solutions.it)
[http://twitter.com/geosolutions_it](http://twitter.com/geosolutions_it)

-------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono
da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate
nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e
-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo
anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.
 
The information in this message and/or attachments, is intended solely for the attention and use of
the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree
June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying,
distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender does not give any warranty or accept liability as the content,
accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which
arise as a result of e-mail transmission, viruses, etc.
4

Hi Niels,

the access info that GeoServer receives from GeoFence may come from more than
one Rule.
The Rule engine looks for all matching rules: constraints contained in Rules
of LIMIT type will be merged so to restrict the constraints found in the ALLOW
rule.
Furthermore, if a user belongs to more than one role, the constraints will be
merged so that the user will be granted all the privileges he should have
according to each role.
All this means that a change in a single Rule may reflect in changes in many
items in the cache, that are quite difficult to track down.
Also, note that the cache is used in both the embedded and the standalone
geofence setup; any changes at that level should take care of not breaking any
of the two setup.
Another point to consider is about making sure this change will also work in a
clustered environment.

   Cheers,
   Emanuele

Alle 13:46:54 di Friday 2 June 2017, Niels Charlier ha scritto:

Hello Nuno,

Another change I have been asked to make in geofence, is that rules in
the cache would automatically invalidated when they are changed. I have
looked at the code, and I have an idea how to do it, and was wondering
if you agree with the approach:

1. write a RuleFilter.matches(Rule rule) method (I think currently rule
filters are only used to translate into queries, and there is no way to
match a filter with a single rule)

2. write a CachedRuleReader.invalidate(Rule rule) which loops through
the whole cache, and invalidates all rule filters that match the rule.

3. write a CachedRuleAdminServiceImpl, that delegates to the normal
RuleAdminServiceImpl but calls the above invalidate upon change of an
existing rule.

Kind Regards

Niels

---------------------------------------------------------------------------
--- Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--

GeoServer Professional Services from the experts!
Visit http://goo.gl/NWWaa2 for more information.

Ing. Emanuele Tajariol
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 380 2116282

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

5

Hello Emanuele,

Thanks for your email.

You say "a change in a single Rule may reflect in changes in many items in the cache, that are quite difficult to track down."

Would a RuleFilter.match(Rule) not be able to track this down? The cache is indexed on rule filters... All rule filters that match the given rule must then be invalidated...

Regards

Niels

On 06-06-17 12:37, Emanuele Tajariol wrote:

Hi Niels,

the access info that GeoServer receives from GeoFence may come from more than
one Rule.
The Rule engine looks for all matching rules: constraints contained in Rules
of LIMIT type will be merged so to restrict the constraints found in the ALLOW
rule.
Furthermore, if a user belongs to more than one role, the constraints will be
merged so that the user will be granted all the privileges he should have
according to each role.
All this means that a change in a single Rule may reflect in changes in many
items in the cache, that are quite difficult to track down.
Also, note that the cache is used in both the embedded and the standalone
geofence setup; any changes at that level should take care of not breaking any
of the two setup.
Another point to consider is about making sure this change will also work in a
clustered environment.

    Cheers,
    Emanuele

Alle 13:46:54 di Friday 2 June 2017, Niels Charlier ha scritto:

Hello Nuno,

Another change I have been asked to make in geofence, is that rules in
the cache would automatically invalidated when they are changed. I have
looked at the code, and I have an idea how to do it, and was wondering
if you agree with the approach:

1. write a RuleFilter.matches(Rule rule) method (I think currently rule
filters are only used to translate into queries, and there is no way to
match a filter with a single rule)

2. write a CachedRuleReader.invalidate(Rule rule) which loops through
the whole cache, and invalidates all rule filters that match the rule.

3. write a CachedRuleAdminServiceImpl, that delegates to the normal
RuleAdminServiceImpl but calls the above invalidate upon change of an
existing rule.

Kind Regards

Niels

---------------------------------------------------------------------------
--- Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

6

Hi Niels,

Would a RuleFilter.match(Rule) not be able to track this down? The cache
is indexed on rule filters... All rule filters that match the given rule
must then be invalidated...

Still not sure about this :slight_smile:
Let's say we change the priority of a Rule; the outcome of an authorization
request will change not only according to the Rule we updated, but also
according to the new Rules that will be selected according to the new
prioritization.

   Cheers,
   Emanuele

Alle 12:42:20 di Tuesday 6 June 2017, Niels Charlier ha scritto:

Hello Emanuele,

Thanks for your email.

You say "a change in a single Rule may reflect in changes in many items
in the cache, that are quite difficult to track down."

Would a RuleFilter.match(Rule) not be able to track this down? The cache
is indexed on rule filters... All rule filters that match the given rule
must then be invalidated...

Regards

Niels

On 06-06-17 12:37, Emanuele Tajariol wrote:
> Hi Niels,
>
> the access info that GeoServer receives from GeoFence may come from more
> than one Rule.
> The Rule engine looks for all matching rules: constraints contained in
> Rules of LIMIT type will be merged so to restrict the constraints found
> in the ALLOW rule.
> Furthermore, if a user belongs to more than one role, the constraints
> will be merged so that the user will be granted all the privileges he
> should have according to each role.
> All this means that a change in a single Rule may reflect in changes in
> many items in the cache, that are quite difficult to track down.
> Also, note that the cache is used in both the embedded and the standalone
> geofence setup; any changes at that level should take care of not
> breaking any of the two setup.
> Another point to consider is about making sure this change will also work
> in a clustered environment.
>
> Cheers,
> Emanuele
>
> Alle 13:46:54 di Friday 2 June 2017, Niels Charlier ha scritto:
>> Hello Nuno,
>>
>> Another change I have been asked to make in geofence, is that rules in
>> the cache would automatically invalidated when they are changed. I have
>> looked at the code, and I have an idea how to do it, and was wondering
>> if you agree with the approach:
>>
>> 1. write a RuleFilter.matches(Rule rule) method (I think currently rule
>> filters are only used to translate into queries, and there is no way to
>> match a filter with a single rule)
>>
>> 2. write a CachedRuleReader.invalidate(Rule rule) which loops through
>> the whole cache, and invalidates all rule filters that match the rule.
>>
>> 3. write a CachedRuleAdminServiceImpl, that delegates to the normal
>> RuleAdminServiceImpl but calls the above invalidate upon change of an
>> existing rule.
>>
>>
>> Kind Regards
>>
>> Niels
>>
>>
>> ------------------------------------------------------------------------
>> --- --- Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Geoserver-devel mailing list
>> Geoserver-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--

GeoServer Professional Services from the experts!
Visit http://goo.gl/NWWaa2 for more information.

Ing. Emanuele Tajariol
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 380 2116282

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

7

Ouch. So do you reckon there is possible method to determine whether a cache entry is affected by a specific rule?

Could we say that my method works as long as priority doesn't change? Or are there other concerns apart from priority?

And in case the priority changes, could we perhaps repeat the process by any rules affected by the priority change? For example
we change a priority from x to y, we repeat the process for all rules from position x to y.

Regards
Niels

On 06-06-17 14:01, Emanuele Tajariol wrote:

Hi Niels,

Would a RuleFilter.match(Rule) not be able to track this down? The cache
is indexed on rule filters... All rule filters that match the given rule
must then be invalidated...

Still not sure about this :slight_smile:
Let's say we change the priority of a Rule; the outcome of an authorization
request will change not only according to the Rule we updated, but also
according to the new Rules that will be selected according to the new
prioritization.

    Cheers,
    Emanuele

Alle 12:42:20 di Tuesday 6 June 2017, Niels Charlier ha scritto:

Hello Emanuele,

Thanks for your email.

You say "a change in a single Rule may reflect in changes in many items
in the cache, that are quite difficult to track down."

Would a RuleFilter.match(Rule) not be able to track this down? The cache
is indexed on rule filters... All rule filters that match the given rule
must then be invalidated...

Regards

Niels

On 06-06-17 12:37, Emanuele Tajariol wrote:

Hi Niels,

the access info that GeoServer receives from GeoFence may come from more
than one Rule.
The Rule engine looks for all matching rules: constraints contained in
Rules of LIMIT type will be merged so to restrict the constraints found
in the ALLOW rule.
Furthermore, if a user belongs to more than one role, the constraints
will be merged so that the user will be granted all the privileges he
should have according to each role.
All this means that a change in a single Rule may reflect in changes in
many items in the cache, that are quite difficult to track down.
Also, note that the cache is used in both the embedded and the standalone
geofence setup; any changes at that level should take care of not
breaking any of the two setup.
Another point to consider is about making sure this change will also work
in a clustered environment.

     Cheers,
     Emanuele

Alle 13:46:54 di Friday 2 June 2017, Niels Charlier ha scritto:

Hello Nuno,

Another change I have been asked to make in geofence, is that rules in
the cache would automatically invalidated when they are changed. I have
looked at the code, and I have an idea how to do it, and was wondering
if you agree with the approach:

1. write a RuleFilter.matches(Rule rule) method (I think currently rule
filters are only used to translate into queries, and there is no way to
match a filter with a single rule)

2. write a CachedRuleReader.invalidate(Rule rule) which loops through
the whole cache, and invalidates all rule filters that match the rule.

3. write a CachedRuleAdminServiceImpl, that delegates to the normal
RuleAdminServiceImpl but calls the above invalidate upon change of an
existing rule.

Kind Regards

Niels

------------------------------------------------------------------------
--- --- Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

8

Could we say that my method works as long as priority doesn't change? Or
are there other concerns apart from priority?

As said, the Rules behaviour is quite contextual, as the Rules work as a whole
and not singularly; the priority example is just one way to make this clear.
Let me show another case: if we are updating a Rule by changing the layer name
field (from L1 to L2), we'll have to invalidate all the entries referring to
either Layer, since this change will modify the rulesets for both layers.
Also, user/role relationship is evaluated within the rule engine, since
GeoServer is only sending out the username when asking for an authorization
check; if a Rule is updated so that the role field changes, you should find out
all the users for both roles (the new one and the old one) and invalidate all
the entries referring to them.

All in all, I don't exclude there is a way to identify the cache entries that
should be invalidated, but I guess that it can be quite hard to implement, and
to debug as well, should any problem arise.

3. write a CachedRuleAdminServiceImpl, that delegates to the normal
RuleAdminServiceImpl but calls the above invalidate upon change of an
existing rule.

Again on this point: please consider that the RuleAdminServiceImpl is a
service which is on GeoFence side
https://github.com/geoserver/geofence/blob/master/src/services/core/services-
impl/src/main/java/org/geoserver/geofence/services/RuleAdminServiceImpl.java

while the rule cache is implemented inside the geoserver plugin:
https://github.com/geoserver/geoserver/blob/master/src/community/geofence/src/main/java/org/geoserver/geofence/cache/CachedRuleReader.java

These two classes are inside the same VM only when geofence is installed as an
embedded service in geoserver. The classes inside the geoserver plugin know
about how to send requests to GeoFence, but, especially in a clustered
environment, GeoFence may not know how to call the services in the geoserver
instances which are using geofence as an auth engine.

Anyway, just out of curiosity, what's the usecase for trying to invalidate
cache entries selectively? Are you changing the rules in geofence so very
often that a complete cache invalidation is not feasible at all?

   Cheers,
   Emanuele

Alle 14:17:26 di Tuesday 6 June 2017, Niels Charlier ha scritto:

Ouch. So do you reckon there is possible method to determine whether a
cache entry is affected by a specific rule?

Could we say that my method works as long as priority doesn't change? Or
are there other concerns apart from priority?

And in case the priority changes, could we perhaps repeat the process by
any rules affected by the priority change? For example
we change a priority from x to y, we repeat the process for all rules
from position x to y.

Regards
Niels

On 06-06-17 14:01, Emanuele Tajariol wrote:
> Hi Niels,
>
>> Would a RuleFilter.match(Rule) not be able to track this down? The cache
>> is indexed on rule filters... All rule filters that match the given rule
>> must then be invalidated...
>
> Still not sure about this :slight_smile:
> Let's say we change the priority of a Rule; the outcome of an
> authorization request will change not only according to the Rule we
> updated, but also according to the new Rules that will be selected
> according to the new prioritization.
>
> Cheers,
> Emanuele
>
> Alle 12:42:20 di Tuesday 6 June 2017, Niels Charlier ha scritto:
>> Hello Emanuele,
>>
>> Thanks for your email.
>>
>> You say "a change in a single Rule may reflect in changes in many items
>> in the cache, that are quite difficult to track down."
>>
>> Would a RuleFilter.match(Rule) not be able to track this down? The cache
>> is indexed on rule filters... All rule filters that match the given rule
>> must then be invalidated...
>>
>> Regards
>>
>> Niels
>>
>> On 06-06-17 12:37, Emanuele Tajariol wrote:
>>> Hi Niels,
>>>
>>> the access info that GeoServer receives from GeoFence may come from
>>> more than one Rule.
>>> The Rule engine looks for all matching rules: constraints contained in
>>> Rules of LIMIT type will be merged so to restrict the constraints found
>>> in the ALLOW rule.
>>> Furthermore, if a user belongs to more than one role, the constraints
>>> will be merged so that the user will be granted all the privileges he
>>> should have according to each role.
>>> All this means that a change in a single Rule may reflect in changes in
>>> many items in the cache, that are quite difficult to track down.
>>> Also, note that the cache is used in both the embedded and the
>>> standalone geofence setup; any changes at that level should take care
>>> of not breaking any of the two setup.
>>> Another point to consider is about making sure this change will also
>>> work in a clustered environment.
>>>
>>> Cheers,
>>> Emanuele
>>>
>>> Alle 13:46:54 di Friday 2 June 2017, Niels Charlier ha scritto:
>>>> Hello Nuno,
>>>>
>>>> Another change I have been asked to make in geofence, is that rules in
>>>> the cache would automatically invalidated when they are changed. I
>>>> have looked at the code, and I have an idea how to do it, and was
>>>> wondering if you agree with the approach:
>>>>
>>>> 1. write a RuleFilter.matches(Rule rule) method (I think currently
>>>> rule filters are only used to translate into queries, and there is no
>>>> way to match a filter with a single rule)
>>>>
>>>> 2. write a CachedRuleReader.invalidate(Rule rule) which loops through
>>>> the whole cache, and invalidates all rule filters that match the rule.
>>>>
>>>> 3. write a CachedRuleAdminServiceImpl, that delegates to the normal
>>>> RuleAdminServiceImpl but calls the above invalidate upon change of an
>>>> existing rule.
>>>>
>>>>
>>>> Kind Regards
>>>>
>>>> Niels
>>>>
>>>>
>>>> ----------------------------------------------------------------------
>>>> -- --- --- Check out the vibrant tech community on one of the world's
>>>> most engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>> _______________________________________________
>>>> Geoserver-devel mailing list
>>>> Geoserver-devel@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--

GeoServer Professional Services from the experts!
Visit http://goo.gl/NWWaa2 for more information.

Ing. Emanuele Tajariol
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 380 2116282

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

9

Hello Emanuele,

Thanks for your explanations. I see your points and I will report this back.

One question though: say we go for complete chance invalidation. Then we still have the problem that this is currently only possible per instance of geoserver, right? Do you see a way to invalidate cache for all instances? What do you think are the possibilities?

Regards
Niels

On 06-06-17 15:41, Emanuele Tajariol wrote:

Could we say that my method works as long as priority doesn't change? Or
are there other concerns apart from priority?

As said, the Rules behaviour is quite contextual, as the Rules work as a whole
and not singularly; the priority example is just one way to make this clear.
Let me show another case: if we are updating a Rule by changing the layer name
field (from L1 to L2), we'll have to invalidate all the entries referring to
either Layer, since this change will modify the rulesets for both layers.
Also, user/role relationship is evaluated within the rule engine, since
GeoServer is only sending out the username when asking for an authorization
check; if a Rule is updated so that the role field changes, you should find out
all the users for both roles (the new one and the old one) and invalidate all
the entries referring to them.

All in all, I don't exclude there is a way to identify the cache entries that
should be invalidated, but I guess that it can be quite hard to implement, and
to debug as well, should any problem arise.

3. write a CachedRuleAdminServiceImpl, that delegates to the normal
RuleAdminServiceImpl but calls the above invalidate upon change of an
existing rule.

Again on this point: please consider that the RuleAdminServiceImpl is a
service which is on GeoFence side
https://github.com/geoserver/geofence/blob/master/src/services/core/services-
impl/src/main/java/org/geoserver/geofence/services/RuleAdminServiceImpl.java

while the rule cache is implemented inside the geoserver plugin:
https://github.com/geoserver/geoserver/blob/master/src/community/geofence/src/main/java/org/geoserver/geofence/cache/CachedRuleReader.java

These two classes are inside the same VM only when geofence is installed as an
embedded service in geoserver. The classes inside the geoserver plugin know
about how to send requests to GeoFence, but, especially in a clustered
environment, GeoFence may not know how to call the services in the geoserver
instances which are using geofence as an auth engine.

Anyway, just out of curiosity, what's the usecase for trying to invalidate
cache entries selectively? Are you changing the rules in geofence so very
often that a complete cache invalidation is not feasible at all?

    Cheers,
    Emanuele

Alle 14:17:26 di Tuesday 6 June 2017, Niels Charlier ha scritto:

Ouch. So do you reckon there is possible method to determine whether a
cache entry is affected by a specific rule?

Could we say that my method works as long as priority doesn't change? Or
are there other concerns apart from priority?

And in case the priority changes, could we perhaps repeat the process by
any rules affected by the priority change? For example
we change a priority from x to y, we repeat the process for all rules
from position x to y.

Regards
Niels

On 06-06-17 14:01, Emanuele Tajariol wrote:

Hi Niels,

Would a RuleFilter.match(Rule) not be able to track this down? The cache
is indexed on rule filters... All rule filters that match the given rule
must then be invalidated...

Still not sure about this :slight_smile:
Let's say we change the priority of a Rule; the outcome of an
authorization request will change not only according to the Rule we
updated, but also according to the new Rules that will be selected
according to the new prioritization.

     Cheers,
     Emanuele

Alle 12:42:20 di Tuesday 6 June 2017, Niels Charlier ha scritto:

Hello Emanuele,

Thanks for your email.

You say "a change in a single Rule may reflect in changes in many items
in the cache, that are quite difficult to track down."

Would a RuleFilter.match(Rule) not be able to track this down? The cache
is indexed on rule filters... All rule filters that match the given rule
must then be invalidated...

Regards

Niels

On 06-06-17 12:37, Emanuele Tajariol wrote:

Hi Niels,

the access info that GeoServer receives from GeoFence may come from
more than one Rule.
The Rule engine looks for all matching rules: constraints contained in
Rules of LIMIT type will be merged so to restrict the constraints found
in the ALLOW rule.
Furthermore, if a user belongs to more than one role, the constraints
will be merged so that the user will be granted all the privileges he
should have according to each role.
All this means that a change in a single Rule may reflect in changes in
many items in the cache, that are quite difficult to track down.
Also, note that the cache is used in both the embedded and the
standalone geofence setup; any changes at that level should take care
of not breaking any of the two setup.
Another point to consider is about making sure this change will also
work in a clustered environment.

      Cheers,
      Emanuele

Alle 13:46:54 di Friday 2 June 2017, Niels Charlier ha scritto:

Hello Nuno,

Another change I have been asked to make in geofence, is that rules in
the cache would automatically invalidated when they are changed. I
have looked at the code, and I have an idea how to do it, and was
wondering if you agree with the approach:

1. write a RuleFilter.matches(Rule rule) method (I think currently
rule filters are only used to translate into queries, and there is no
way to match a filter with a single rule)

2. write a CachedRuleReader.invalidate(Rule rule) which loops through
the whole cache, and invalidates all rule filters that match the rule.

3. write a CachedRuleAdminServiceImpl, that delegates to the normal
RuleAdminServiceImpl but calls the above invalidate upon change of an
existing rule.

Kind Regards

Niels

----------------------------------------------------------------------
-- --- --- Check out the vibrant tech community on one of the world's
most engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

10

Hi Niels,

Do you see a way to invalidate cache for all
instances? What do you think are the possibilities?

Some time ago, discussing about GeoFence in a clustered environment, we were
thinking about using a distributed cache, such as ehcache or hazelcast, in
place of the currently used cache implementation.

If the cluster is implemented using jms
(http://docs.geoserver.org/latest/en/user/community/jms-cluster/index.html)
the cache invalidation may be perfomed using jms as well, so that any instance
will invalidate its own cache copy.

   Cheers,
   Emanuele

Alle 15:40:01 di Thursday 8 June 2017, Niels Charlier ha scritto:

Hello Emanuele,

Thanks for your explanations. I see your points and I will report this
back.

One question though: say we go for complete chance invalidation. Then we
still have the problem that this is currently only possible per instance
of geoserver, right? Do you see a way to invalidate cache for all
instances? What do you think are the possibilities?

Regards
Niels

On 06-06-17 15:41, Emanuele Tajariol wrote:
>> Could we say that my method works as long as priority doesn't change? Or
>> are there other concerns apart from priority?
>
> As said, the Rules behaviour is quite contextual, as the Rules work as a
> whole and not singularly; the priority example is just one way to make
> this clear. Let me show another case: if we are updating a Rule by
> changing the layer name field (from L1 to L2), we'll have to invalidate
> all the entries referring to either Layer, since this change will modify
> the rulesets for both layers. Also, user/role relationship is evaluated
> within the rule engine, since GeoServer is only sending out the username
> when asking for an authorization check; if a Rule is updated so that the
> role field changes, you should find out all the users for both roles
> (the new one and the old one) and invalidate all the entries referring
> to them.
>
> All in all, I don't exclude there is a way to identify the cache entries
> that should be invalidated, but I guess that it can be quite hard to
> implement, and to debug as well, should any problem arise.
>
>> 3. write a CachedRuleAdminServiceImpl, that delegates to the normal
>> RuleAdminServiceImpl but calls the above invalidate upon change of an
>> existing rule.
>
> Again on this point: please consider that the RuleAdminServiceImpl is a
> service which is on GeoFence side
> https://github.com/geoserver/geofence/blob/master/src/services/core/servi
> ces-
> impl/src/main/java/org/geoserver/geofence/services/RuleAdminServiceImpl.
> java
>
> while the rule cache is implemented inside the geoserver plugin:
> https://github.com/geoserver/geoserver/blob/master/src/community/geofence
> /src/main/java/org/geoserver/geofence/cache/CachedRuleReader.java
>
> These two classes are inside the same VM only when geofence is installed
> as an embedded service in geoserver. The classes inside the geoserver
> plugin know about how to send requests to GeoFence, but, especially in a
> clustered environment, GeoFence may not know how to call the services in
> the geoserver instances which are using geofence as an auth engine.
>
>
> Anyway, just out of curiosity, what's the usecase for trying to
> invalidate cache entries selectively? Are you changing the rules in
> geofence so very often that a complete cache invalidation is not
> feasible at all?
>
> Cheers,
> Emanuele
>
> Alle 14:17:26 di Tuesday 6 June 2017, Niels Charlier ha scritto:
>> Ouch. So do you reckon there is possible method to determine whether a
>> cache entry is affected by a specific rule?
>>
>> Could we say that my method works as long as priority doesn't change? Or
>> are there other concerns apart from priority?
>>
>> And in case the priority changes, could we perhaps repeat the process by
>> any rules affected by the priority change? For example
>> we change a priority from x to y, we repeat the process for all rules
>> from position x to y.
>>
>> Regards
>> Niels
>>
>> On 06-06-17 14:01, Emanuele Tajariol wrote:
>>> Hi Niels,
>>>
>>>> Would a RuleFilter.match(Rule) not be able to track this down? The
>>>> cache is indexed on rule filters... All rule filters that match the
>>>> given rule must then be invalidated...
>>>
>>> Still not sure about this :slight_smile:
>>> Let's say we change the priority of a Rule; the outcome of an
>>> authorization request will change not only according to the Rule we
>>> updated, but also according to the new Rules that will be selected
>>> according to the new prioritization.
>>>
>>> Cheers,
>>> Emanuele
>>>
>>> Alle 12:42:20 di Tuesday 6 June 2017, Niels Charlier ha scritto:
>>>> Hello Emanuele,
>>>>
>>>> Thanks for your email.
>>>>
>>>> You say "a change in a single Rule may reflect in changes in many
>>>> items in the cache, that are quite difficult to track down."
>>>>
>>>> Would a RuleFilter.match(Rule) not be able to track this down? The
>>>> cache is indexed on rule filters... All rule filters that match the
>>>> given rule must then be invalidated...
>>>>
>>>> Regards
>>>>
>>>> Niels
>>>>
>>>> On 06-06-17 12:37, Emanuele Tajariol wrote:
>>>>> Hi Niels,
>>>>>
>>>>> the access info that GeoServer receives from GeoFence may come from
>>>>> more than one Rule.
>>>>> The Rule engine looks for all matching rules: constraints contained
>>>>> in Rules of LIMIT type will be merged so to restrict the constraints
>>>>> found in the ALLOW rule.
>>>>> Furthermore, if a user belongs to more than one role, the constraints
>>>>> will be merged so that the user will be granted all the privileges he
>>>>> should have according to each role.
>>>>> All this means that a change in a single Rule may reflect in changes
>>>>> in many items in the cache, that are quite difficult to track down.
>>>>> Also, note that the cache is used in both the embedded and the
>>>>> standalone geofence setup; any changes at that level should take
>>>>> care of not breaking any of the two setup.
>>>>> Another point to consider is about making sure this change will also
>>>>> work in a clustered environment.
>>>>>
>>>>> Cheers,
>>>>> Emanuele
>>>>>
>>>>> Alle 13:46:54 di Friday 2 June 2017, Niels Charlier ha scritto:
>>>>>> Hello Nuno,
>>>>>>
>>>>>> Another change I have been asked to make in geofence, is that rules
>>>>>> in the cache would automatically invalidated when they are changed.
>>>>>> I have looked at the code, and I have an idea how to do it, and was
>>>>>> wondering if you agree with the approach:
>>>>>>
>>>>>> 1. write a RuleFilter.matches(Rule rule) method (I think currently
>>>>>> rule filters are only used to translate into queries, and there is
>>>>>> no way to match a filter with a single rule)
>>>>>>
>>>>>> 2. write a CachedRuleReader.invalidate(Rule rule) which loops
>>>>>> through the whole cache, and invalidates all rule filters that
>>>>>> match the rule.
>>>>>>
>>>>>> 3. write a CachedRuleAdminServiceImpl, that delegates to the normal
>>>>>> RuleAdminServiceImpl but calls the above invalidate upon change of
>>>>>> an existing rule.
>>>>>>
>>>>>>
>>>>>> Kind Regards
>>>>>>
>>>>>> Niels
>>>>>>
>>>>>>
>>>>>> --------------------------------------------------------------------
>>>>>> -- -- --- --- Check out the vibrant tech community on one of the
>>>>>> world's most engaging tech sites, Slashdot.org!
>>>>>> http://sdm.link/slashdot
>>>>>> _______________________________________________
>>>>>> Geoserver-devel mailing list
>>>>>> Geoserver-devel@lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--

GeoServer Professional Services from the experts!
Visit http://goo.gl/NWWaa2 for more information.

Ing. Emanuele Tajariol
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 380 2116282

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------