[Geoserver-devel] Extending control flow by config per layer

Hello list,

we would like to use the control flow extension, but we need to configure it per layer, which is currently not supported. We will implement something for our project and we can offer to contribute the code.

1) Is there any interest in such a contribution?

I had a look at the code and I have some questions upfront, trying to create a solution which is benefitial for the GeoServer project:

2) I have experience with WMS and WFS only. For WMS it is quite simple to extract the layers from the URL. The GetMap implementation uses a GetMapKvpRequestReader to extract the layers, which is part of gs-wms. This also performs some validations but doesnt provide an API to extract the layers only. I suggest not to use the GetMapKvpRequestReader but simply check the HTTP request param "layers". Is this approach acceptable? Any other recommendations?

3) To determine the affected WFS feature type is not so simple. The information might be contained in an XML document, which is not parsed at that point AFAICS. Duplicate parsing is not a solution. I currently dont see a simple solution. In our project we need to restrict GetMap requests only, so I will concentrate on a solution for WMS GetMap and leave the rest open. Is this acceptable and still of worth for the community? I would log some warnings if somebody configured a limit for a layer using service-request-combination other than WMS-GetMap.

4) The configuration is currently as follows:
ows.<service>[.<request>[.<outputFormat>]]=<count>

4.1)
I suggest to extend it to
ows.<service>[.<request>[.<outputFormat>[.<layer>]]]=<count>

Actually the ordering is not quite as expected, it would be better to keep it top-down (ows.<service>[.<request>[.<layer>[.<outputFormat>]]]), but that would be a breaking change in the configuration.

4.2)
As values for <outputFormat> I'd support an asterisk (*), matching all formats.

4.3)
As value for layer I'd support an asterisk, too, for text matching, for example to match a layers of a workspace (tiger:*)

Any comments are welcome.

Best regards,
Andreas

On Mon, Jan 15, 2018 at 11:13 AM, Watermeyer, Andreas <
Andreas.Watermeyer@anonymised.com> wrote:

Hello list,

we would like to use the control flow extension, but we need to configure
it per layer, which is currently not supported. We will implement something
for our project and we can offer to contribute the code.

1) Is there any interest in such a contribution?

Yes, that seems to be a useful addition, at least in principle. What's the
use case for doing it by layer? Typically I hear of users asking different
rules per user role instead (QoS classes).
Also, how do you handle requests with multiple layers? Most strict rule
applies?

I had a look at the code and I have some questions upfront, trying to
create a solution which is benefitial for the GeoServer project:

2) I have experience with WMS and WFS only. For WMS it is quite simple to
extract the layers from the URL. The GetMap implementation uses
a GetMapKvpRequestReader to extract the layers, which is part of gs-wms.
This also performs some validations but doesnt provide an API to extract
the layers only. I suggest not to use the GetMapKvpRequestReader but simply
check the HTTP request param "layers". Is this approach acceptable? Any
other recommendations?

3) To determine the affected WFS feature type is not so simple. The
information might be contained in an XML document, which is not parsed at
that point AFAICS. Duplicate parsing is not a solution. I currently dont
see a simple solution. In our project we need to restrict GetMap requests
only, so I will concentrate on a solution for WMS GetMap and leave the rest
open. Is this acceptable and still of worth for the community? I would log
some warnings if somebody configured a limit for a layer using
service-request-combination other than WMS-GetMap.

Doing your own parsing is indeed not an option. Check what the monitoring
module is doing, it's already extracting layer lists from the various
protocols, using a dedicated class fetching
the info post-parsing.

4) The configuration is currently as follows:
ows.<service>[.<request>[.<outputFormat>]]=<count>

4.1)
I suggest to extend it to
ows.<service>[.<request>[.<outputFormat>[.<layer>]]]=<count>

Actually the ordering is not quite as expected, it would be better to keep
it top-down (ows.<service>[.<request>[.<layer>[.<outputFormat>]]]), but
that would be a breaking change in the configuration.

Yeah, it's not exactly great. Thinking out loud, maybe it's time to switch
it to a XML format. Others might chime in with ideas too.

4.2)
As values for <outputFormat> I'd support an asterisk (*), matching all
formats.

4.3)
As value for layer I'd support an asterisk, too, for text matching, for
example to match a layers of a workspace (tiger:*)

Yep, the above is consistent with other usage in GeoServer (e.g., security)

Cheers
Andrea

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V
for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

Hi Andrea,

thanks for your quick response! See my answers below inline.

···

Best regards,
Andreas


Von: andrea.aime@anonymised.com andrea.aime@anonymised.com im Auftrag von Andrea Aime <andrea.aime@anonymised.com.1268…>
Gesendet: Montag, 15. Januar 2018 11:46
An: Watermeyer, Andreas
Cc: geoserver-devel@lists.sourceforge.net
Betreff: Re: [Geoserver-devel] Extending control flow by config per layer

On Mon, Jan 15, 2018 at 11:13 AM, Watermeyer, Andreas <Andreas.Watermeyer@anonymised.com> wrote:

Hello list,

we would like to use the control flow extension, but we need to configure it per layer, which is currently not supported. We will implement something for our project and we can offer to contribute the code.

  1. Is there any interest in such a contribution?

Yes, that seems to be a useful addition, at least in principle. What’s the use case for doing it by layer? Typically I hear of users asking different rules per user role instead (QoS classes).

Also, how do you handle requests with multiple layers? Most strict rule applies?

We have two kinds of layers: The first kind is based on Oracle, GeoServer renders the images. The second is just cascading another WMS, GeoServer enforcess access control rules. In our case only the first shall be throttled in order to prevent CPU bottlenecks through rendering. Thats what I understood from several recommendations (your “GeoServer in production” slides, etc).

Multiple layers: It would be as in the current implementation: The flow controllers would processed by one by one, ordered by priority. If multiple match, multiple have to be satisfied. I wouldnt skip one controller because of the match of another.

I had a look at the code and I have some questions upfront, trying to create a solution which is benefitial for the GeoServer project:

  1. I have experience with WMS and WFS only. For WMS it is quite simple to extract the layers from the URL. The GetMap implementation uses a GetMapKvpRequestReader to extract the layers, which is part of gs-wms. This also performs some validations but doesnt provide an API to extract the layers only. I suggest not to use the GetMapKvpRequestReader but simply check the HTTP request param “layers”. Is this approach acceptable? Any other recommendations?

  2. To determine the affected WFS feature type is not so simple. The information might be contained in an XML document, which is not parsed at that point AFAICS. Duplicate parsing is not a solution. I currently dont see a simple solution. In our project we need to restrict GetMap requests only, so I will concentrate on a solution for WMS GetMap and leave the rest open. Is this acceptable and still of worth for the community? I would log some warnings if somebody configured a limit for a layer using service-request-combination other than WMS-GetMap.

Doing your own parsing is indeed not an option. Check what the monitoring module is doing, it’s already extracting layer lists from the various protocols, using a dedicated class fetching

the info post-parsing.

That is interesting. Now I see the operation specific parameter object is already there, so parsing etc has already taken place. The control flow would require the same code as the monitoring extension. What do you think about moving the code to a common module available for both? Which?

  1. The configuration is currently as follows:
    ows.[.[.]]=

4.1)
I suggest to extend it to
ows.[.[.[.]]]=

Actually the ordering is not quite as expected, it would be better to keep it top-down (ows.[.[.[.]]]), but that would be a breaking change in the configuration.

Yeah, it’s not exactly great. Thinking out loud, maybe it’s time to switch it to a XML format. Others might chime in with ideas too.

4.2)
As values for I’d support an asterisk (*), matching all formats.

4.3)
As value for layer I’d support an asterisk, too, for text matching, for example to match a layers of a workspace (tiger:*)

Yep, the above is consistent with other usage in GeoServer (e.g., security)

Cheers
Andrea

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.