[Geoserver-devel] Process selection landed on trunk

Hi,
I’ve just landed the process selection work on trunk

Compared to the original patch the are the following changes:

  • the labels in the WPS Admin page have been changed a bit following Justin’s suggestions
  • there is a new column reporting the prefix(es) used by the processes in that group,
    following Martin’s suggestion
  • the wps design guide page in the dev guide contains a short description with a few
    pointers to the usage of ProcessFilter

The GUI now looks like this:

Inline image 1

Ugh, two factories with the same prefix, ugly indeed… afaik Justin is going to push for a cleanup
of the factory structure back in GeoTools before the process API gets pushed to supported land
in GeoTools.

About the documentation for the process listeners, it’s not much, but generally much more
than for all the other extension points we have around, like dispatcher callbacks, transaction listeners,
output formats of various kinds: is there a plan to add such documentation?
Would certainly be welcomed by people trying to extend GeoServer.

Cheers
Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


A couple of comments:

  • The name ProcessBlacklistSelector to me doesn’t indicate what it’s actually doing. Could it be named more descriptively, say UnsupportedParameterTypeProcessFilter?

  • Since the immediate use case is apparently for blacklisting, I’m wondering whether a UI based on a list of process names would be simpler than the checkbox-based UI? That way the Admin can easily see which processes are disabled. A shorthand could be provided for disabling entire namespaces. This would sidestep the issue with grouping and Wicket, too.

On Mon, Aug 13, 2012 at 10:29 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
I’ve just landed the process selection work on trunk

Compared to the original patch the are the following changes:

  • the labels in the WPS Admin page have been changed a bit following Justin’s suggestions
  • there is a new column reporting the prefix(es) used by the processes in that group,
    following Martin’s suggestion
  • the wps design guide page in the dev guide contains a short description with a few
    pointers to the usage of ProcessFilter

The GUI now looks like this:

Inline image 1

Ugh, two factories with the same prefix, ugly indeed… afaik Justin is going to push for a cleanup
of the factory structure back in GeoTools before the process API gets pushed to supported land
in GeoTools.

About the documentation for the process listeners, it’s not much, but generally much more
than for all the other extension points we have around, like dispatcher callbacks, transaction listeners,
output formats of various kinds: is there a plan to add such documentation?
Would certainly be welcomed by people trying to extend GeoServer.

Cheers
Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it



Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Martin Davis
OpenGeo - http://opengeo.org
Expert service straight from the developers.

On Mon, Aug 13, 2012 at 8:53 PM, Martin Davis <mdavis@anonymised.com> wrote:

A couple of comments:

  • The name ProcessBlacklistSelector to me doesn’t indicate what it’s actually doing. Could it be named more descriptively, say UnsupportedParameterTypeProcessFilter?

I can do that.

  • Since the immediate use case is apparently for blacklisting, I’m wondering whether a UI based on a list of process names would be simpler than the checkbox-based UI? That way the Admin can easily see which processes are disabled. A shorthand could be provided for disabling entire namespaces. This would sidestep the issue with grouping and Wicket, too.

Do you mean, a textbox that one has to fill with the processes he wants to disable?
That does not seem usable at all to me, one has to know the process names by heart and
would have to painfully write them one by one.
Granted, it’s cheap to write (would have been much quicker to put togheter),
but I would not be happy to have to use something like that, neither I believe would be the average user.

I expect that a normal WPS will have 90% of the processes disabled once in production, since WPS
is heavy you want to reduce to the maximum possible the amount of processes exposed to
the net.
My first attempt was to make a single table, but there are many processes, and many more can
be added along the way, so that would need paging, but as I said before we never managed to
have a table that is both selectable and pageable (doing it is possible, but not entirely trivial,
and would work only for lists that have relatively little entries before becoming memory bound).

With the GUI I’ve put together getting only those 5 processes you really want to expose enabled
is quick, disable the factories you don’t want, then enter in the one remaining, disable all
of the processes in it (clicking on the checkbox at top), and re-enable only the ones you want.

Cheers
Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


On Mon, Aug 13, 2012 at 12:55 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Aug 13, 2012 at 8:53 PM, Martin Davis <mdavis@anonymised.com> wrote:

A couple of comments:

  • The name ProcessBlacklistSelector to me doesn’t indicate what it’s actually doing. Could it be named more descriptively, say UnsupportedParameterTypeProcessFilter?

I can do that.

Great!

  • Since the immediate use case is apparently for blacklisting, I’m wondering whether a UI based on a list of process names would be simpler than the checkbox-based UI? That way the Admin can easily see which processes are disabled. A shorthand could be provided for disabling entire namespaces. This would sidestep the issue with grouping and Wicket, too.

Do you mean, a textbox that one has to fill with the processes he wants to disable?
That does not seem usable at all to me, one has to know the process names by heart and
would have to painfully write them one by one.
Granted, it’s cheap to write (would have been much quicker to put togheter),
but I would not be happy to have to use something like that, neither I believe would be the average user.

Yes, that’s what I meant. A Blacklist textbox that accepted process names (“gs:Buffer”), with wildcards for namespace (“gs:“)and all processes (””). Actually, having both a whitelist and blacklist box would be even more flexible - the whitelist would list enabled processes, and the blacklist would subtract from the whitelist for fine-tuning.

My expectation is that if an admin is wanting to limit process to just a few, or remove a few undesirable processes, they are going to know very well which ones those are. And cut-and-paste and wildcards make it pretty easy to build the lists. Other advantages include being able to easily copy lists, which assists in documentation and in replaying into other instances.

I expect that a normal WPS will have 90% of the processes disabled once in production, since WPS
is heavy you want to reduce to the maximum possible the amount of processes exposed to
the net.

I’m not so sure this is the only or typical use case. If the intent is to use GeoServer as a general processing server, it is probably desirable to expose all of the “standard” processes (whatever that list ends up being). If the intent is to say expose only geometry-based processes, then the wildcard list approach lets you do that easily.

My first attempt was to make a single table, but there are many processes, and many more can
be added along the way, so that would need paging, but as I said before we never managed to
have a table that is both selectable and pageable (doing it is possible, but not entirely trivial,
and would work only for lists that have relatively little entries before becoming memory bound).

Yes, understood that Wicket puts some constraints on UI design. The list approach avoids this as well.

With the GUI I’ve put together getting only those 5 processes you really want to expose enabled
is quick, disable the factories you don’t want, then enter in the one remaining, disable all
of the processes in it (clicking on the checkbox at top), and re-enable only the ones you want.

Cheers

Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


Martin Davis
OpenGeo - http://opengeo.org
Expert service straight from the developers.

On Mon, Aug 13, 2012 at 10:36 PM, Martin Davis <mdavis@anonymised.com> wrote:

Yes, that’s what I meant. A Blacklist textbox that accepted process names (“gs:Buffer”), with wildcards for namespace (“gs:“)and all processes (””). Actually, having both a whitelist and blacklist box would be even more flexible - the whitelist would list enabled processes, and the blacklist would subtract from the whitelist for fine-tuning.

My expectation is that if an admin is wanting to limit process to just a few, or remove a few undesirable processes, they are going to know very well which ones those are. And cut-and-paste and wildcards make it pretty easy to build the lists. Other advantages include being able to easily copy lists, which assists in documentation and in replaying into other instances.

If this is considered a good UI for the typical GeoServer user then probably the typical GS user is coming from
the MapServer mapfiles.

Having a whitelist is something I considered as well, but it would put a wrench in scripting approaches, you write
a script and then you also have to manually have to configure it in the UI, that does not sound like a good approach
to me. You may say such user only has to add a “python:*” directive, I still believe that’s too much of a text UI
for the GeoServer target audience.

Let’s put it like this: we have a -1 from me on such a UI, and a +1 from you of course.
Let’s hear from others, if textbox approach gets a majority feel free to implement it and scrap mine :slight_smile:

Cheers
Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


On Mon, Aug 13, 2012 at 2:58 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Aug 13, 2012 at 10:36 PM, Martin Davis <mdavis@anonymised.com> wrote:

Yes, that’s what I meant. A Blacklist textbox that accepted process names (“gs:Buffer”), with wildcards for namespace (“gs:“)and all processes (””). Actually, having both a whitelist and blacklist box would be even more flexible - the whitelist would list enabled processes, and the blacklist would subtract from the whitelist for fine-tuning.

My expectation is that if an admin is wanting to limit process to just a few, or remove a few undesirable processes, they are going to know very well which ones those are. And cut-and-paste and wildcards make it pretty easy to build the lists. Other advantages include being able to easily copy lists, which assists in documentation and in replaying into other instances.

If this is considered a good UI for the typical GeoServer user then probably the typical GS user is coming from
the MapServer mapfiles.

Having a whitelist is something I considered as well, but it would put a wrench in scripting approaches, you write
a script and then you also have to manually have to configure it in the UI, that does not sound like a good approach
to me. You may say such user only has to add a “python:*” directive, I still believe that’s too much of a text UI
for the GeoServer target audience.

Let’s put it like this: we have a -1 from me on such a UI, and a +1 from you of course.
Let’s hear from others, if textbox approach gets a majority feel free to implement it and scrap mine :slight_smile:

I can actually see the argument for both approaches. I think the current list/table based way is more typical of how the geoserver configuration works. There does however seem to be room for improvements like adding searching, paging, etc…

In the end we don’t really have a ui “style guide” so there is no precedent. What one dev thinks is usable is probably radically different from what another dev thinks is usable. Both of which probably differ from what a real ux person thinks :slight_smile:

Cheers

Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it



Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

On Mon, Aug 13, 2012 at 1:58 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Aug 13, 2012 at 10:36 PM, Martin Davis <mdavis@anonymised.com> wrote:

Yes, that’s what I meant. A Blacklist textbox that accepted process names (“gs:Buffer”), with wildcards for namespace (“gs:“)and all processes (””). Actually, having both a whitelist and blacklist box would be even more flexible - the whitelist would list enabled processes, and the blacklist would subtract from the whitelist for fine-tuning.

My expectation is that if an admin is wanting to limit process to just a few, or remove a few undesirable processes, they are going to know very well which ones those are. And cut-and-paste and wildcards make it pretty easy to build the lists. Other advantages include being able to easily copy lists, which assists in documentation and in replaying into other instances.

If this is considered a good UI for the typical GeoServer user then probably the typical GS user is coming from
the MapServer mapfiles.

Having a whitelist is something I considered as well, but it would put a wrench in scripting approaches, you write
a script and then you also have to manually have to configure it in the UI, that does not sound like a good approach
to me. You may say such user only has to add a “python:*” directive, I still believe that’s too much of a text UI
for the GeoServer target audience.

The wildcarding should make this easy. The default configuration would have “*” in the whitelist, and nothing in the blacklist. This makes all new processes visible by default.

If a more restrictive whitelist was in place, and new processes were being added in a given namespace “ns”, then the admin simply has to add “ns:*” to the whitelist.

Let’s put it like this: we have a -1 from me on such a UI, and a +1 from you of course.
Let’s hear from others, if textbox approach gets a majority feel free to implement it and scrap mine :slight_smile:

Would be interesting to hear input from other GeoServer devs and admins. Or maybe just implement it and get the feedback later. 8^)

Cheers

Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


Martin Davis
OpenGeo - http://opengeo.org
Expert service straight from the developers.

On Tue, Aug 14, 2012 at 1:59 AM, Martin Davis <mdavis@anonymised.com> wrote:

Andrea, I just tried this out, but I think I hit a few bumps:

  • On the Process Selection page I expected to see the list of processes sorted by name, but this doesn’t appear to be the case. It’s hard to find processes if they aren’t sorted.

No problem, that’s easily fixed

  • Is the WPS Request Builder supposed to respect the blacklist? Didn’t seem to be working for me

Did you press “save” on the wps admin page? The process selection pages are secondary ones, like the SQL
view ones, you have to save in the main one for the changes to be applied.
I tested it out before sending the first mail, it worked for me
(I wanted to include a screenshot of the request builder with only 4 processes, but it turned out it’s impossible
to do that and keep the drop-down opened at the same time in Linux)

  • The GeoServerProcessors applyFilters method gets called a LOT - e.g. numerous times during constructing a single RequestBuilder page. It seems like if there is a blacklist in place, then it will be evaluated every time this is called, with the creation of a new SelectingProcessFactory each time? Will this be an issue for performance?

The RequestBuilder page is a non use case performance wise, it’s not a proper WPS client and it only gets used by umans.
What could be worrysome is the code path of the OGC requests, that may end up handling hundred of requests
by second, however processes are normally quite time consuming compared to the average OGC request,
so I’m not that worried there either.

Introducing caching in the current design may not be that easy:

  • the factories can be dynamic in the processes they do return
  • the list of filtered processes can change over time
  • the filter themselves can wrap the original factories and introduce
    dynamic behavior based on the user authenticated in the current
    request thread

That said, I believe it’s possible to create a cache of SelectingProcessFactory
based on the original class (derived by calling getInnermostDelegate on
DelegatingProcessFactory), and then listen to WPS re-configuration events
and drop the cache to make it respect the new rules.
Want to have a crack at it?

CheersAndrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


On Mon, Aug 13, 2012 at 11:45 PM, Martin Davis <mdavis@anonymised.com> wrote:

The wildcarding should make this easy. The default configuration would have “*” in the whitelist, and nothing in the blacklist. This makes all new processes visible by default.

If a more restrictive whitelist was in place, and new processes were being added in a given namespace “ns”, then the admin simply has to add “ns:*” to the whitelist.

I don’t see anything simple in having to remember by heart prefixes and process names, in particular
I find very obnoxious the “limited srs lilst” thing in wms page exactly because it forces one to have
to remember by heart the list of EPSG codes one wants to activate.
I guess some have a much better memory than me, but if the only UI available would be the one you
say in order to fill that box I’d have to grab the WPS capabilities document and start copying and pasting
the names of the processes I want (out of a XML document, ugh), doing it by heart I’d be sure to
introduce typos and other little errors in the list.

Regarding the while versus black list, the UI I’ve made is very explicit, you see all processes and you
can check on and off the ones you prefer, what makes it a blacklist is the behavior against unknown
processes, if a new process is not in the list it’s allowed to be exposed.

If having a whitelist approach is deemed to be important we can have a checkbox on top of the
groups table stating “Automatically enable new processes” that the admin can check on and off,
and saving in WPSInfo not only the list of the processes that are to be filtered out, but also the
one of the enabled processes, and the checkbox will deal wiith the rest.

Let’s put it like this: we have a -1 from me on such a UI, and a +1 from you of course.
Let’s hear from others, if textbox approach gets a majority feel free to implement it and scrap mine :slight_smile:

Would be interesting to hear input from other GeoServer devs and admins. Or maybe just implement it and get the feedback later. 8^)

The latter would be rather hideous, a developer overwriting another active developer work just based on his
personal judgement.
That’s not how we work around here, we try to reach consensus instead, even if sometimes it is a painful
one to reach.

As said, let’s have others take a position as well if they think one solution is better than the other, if
we don’t get any development I’d suggest to keep the implementation we already have.

Cheers
Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


On Tue, Aug 14, 2012 at 8:08 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Tue, Aug 14, 2012 at 1:59 AM, Martin Davis <mdavis@anonymised.com> wrote:

Andrea, I just tried this out, but I think I hit a few bumps:

  • On the Process Selection page I expected to see the list of processes sorted by name, but this doesn’t appear to be the case. It’s hard to find processes if they aren’t sorted.

No problem, that’s easily fixed

Done

  • Is the WPS Request Builder supposed to respect the blacklist? Didn’t seem to be working for me

Did you press “save” on the wps admin page? The process selection pages are secondary ones, like the SQL
view ones, you have to save in the main one for the changes to be applied.
I tested it out before sending the first mail, it worked for me
(I wanted to include a screenshot of the request builder with only 4 processes, but it turned out it’s impossible
to do that and keep the drop-down opened at the same time in Linux)

Just double checked, it works the moment you submit the wps admin page.
Maybe the process selection sub-page should have a “back” button instead of a “apply” one?

In the beginning we experimented having dialog boxes in the UI, which make it easier to understand there
is no immediate action taken when you submit them, but they turned out to be
generally ugly and hard to control, besides in this case the amount of info to be displayed would not fit
the space of one (they have to be sized in pixels and we have no way to predict how big the user
screen will be, so we have to keep them small).

Cheers
Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


On Mon, Aug 13, 2012 at 8:53 PM, Martin Davis <mdavis@anonymised.com> wrote:

A couple of comments:

  • The name ProcessBlacklistSelector to me doesn’t indicate what it’s actually doing. Could it be named more descriptively, say UnsupportedParameterTypeProcessFilter?

Renamed this one as well (including the reference in the docs)

Cheers
Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


It seems Martin is more interested in power (the ability to white- and black-list individual processes while still being able to have blanket/wildcard rules) and Andrea is more intrested in accessibility (via a point-and-click interface.) As I understand it the current UI defaults to publishing a rule and allows the admin to blacklist blocks of process by factory (and not namespace.)

Instead of switching to a fully text-based interface, perhaps we could make the GUI more sophisticated without sacrificing too much accessibility?

One thought I had was the idea of a more generic “rule” concept - instead of a rule consisting of the pairing of a namespace and a allow/disallow flag, maybe rules could also include a “mode” indicating whether they apply to a namespace or an individual process. A dropdown or auto-completing text field could assist the user in filling out the namespace/process field. I attached a little mockup I made of what that might look like (with a “preview” thrown in at the bottom showing blocked and allowed processes explicitly.)

Or we could have expanders in the rule editing control - maybe each namespace has a little “+” by it to allow making exceptions (so for a blocked namespace, expanding it allows you to choose individual processes to allow, or the other way around for allowed namespaces.)

Depending on how it’s done, the GUI could even persist to a plain text file that MapServer-style admins could hack on :slight_smile:


David Winslow
OpenGeo - http://opengeo.org/

On Tue, Aug 14, 2012 at 2:18 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Aug 13, 2012 at 11:45 PM, Martin Davis <mdavis@anonymised.com> wrote:

The wildcarding should make this easy. The default configuration would have “*” in the whitelist, and nothing in the blacklist. This makes all new processes visible by default.

If a more restrictive whitelist was in place, and new processes were being added in a given namespace “ns”, then the admin simply has to add “ns:*” to the whitelist.

I don’t see anything simple in having to remember by heart prefixes and process names, in particular
I find very obnoxious the “limited srs lilst” thing in wms page exactly because it forces one to have
to remember by heart the list of EPSG codes one wants to activate.
I guess some have a much better memory than me, but if the only UI available would be the one you
say in order to fill that box I’d have to grab the WPS capabilities document and start copying and pasting
the names of the processes I want (out of a XML document, ugh), doing it by heart I’d be sure to
introduce typos and other little errors in the list.

Regarding the while versus black list, the UI I’ve made is very explicit, you see all processes and you
can check on and off the ones you prefer, what makes it a blacklist is the behavior against unknown
processes, if a new process is not in the list it’s allowed to be exposed.

If having a whitelist approach is deemed to be important we can have a checkbox on top of the
groups table stating “Automatically enable new processes” that the admin can check on and off,
and saving in WPSInfo not only the list of the processes that are to be filtered out, but also the
one of the enabled processes, and the checkbox will deal wiith the rest.

Let’s put it like this: we have a -1 from me on such a UI, and a +1 from you of course.
Let’s hear from others, if textbox approach gets a majority feel free to implement it and scrap mine :slight_smile:

Would be interesting to hear input from other GeoServer devs and admins. Or maybe just implement it and get the feedback later. 8^)

The latter would be rather hideous, a developer overwriting another active developer work just based on his
personal judgement.
That’s not how we work around here, we try to reach consensus instead, even if sometimes it is a painful
one to reach.

As said, let’s have others take a position as well if they think one solution is better than the other, if
we don’t get any development I’d suggest to keep the implementation we already have.

Cheers

Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it



Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

(attachments)

process-rules.png

On Tue, Aug 14, 2012 at 4:21 PM, David Winslow <dwinslow@anonymised.com> wrote:

It seems Martin is more interested in power (the ability to white- and black-list individual processes while still being able to have blanket/wildcard rules) and Andrea is more intrested in accessibility (via a point-and-click interface.) As I understand it the current UI defaults to publishing a rule and allows the admin to blacklist blocks of process by factory (and not namespace.)

Instead of switching to a fully text-based interface, perhaps we could make the GUI more sophisticated without sacrificing too much accessibility?

One thought I had was the idea of a more generic “rule” concept - instead of a rule consisting of the pairing of a namespace and a allow/disallow flag, maybe rules could also include a “mode” indicating whether they apply to a namespace or an individual process. A dropdown or auto-completing text field could assist the user in filling out the namespace/process field. I attached a little mockup I made of what that might look like (with a “preview” thrown in at the bottom showing blocked and allowed processes explicitly.)

Or we could have expanders in the rule editing control - maybe each namespace has a little “+” by it to allow making exceptions (so for a blocked namespace, expanding it allows you to choose individual processes to allow, or the other way around for allowed namespaces.)

This would work for me, but it seems it’s a ton of work, especially UI wise, but it would also.
Who’s going to do something like this, and with what timings?

Cheers
Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


On Mon, Aug 13, 2012 at 11:08 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Tue, Aug 14, 2012 at 1:59 AM, Martin Davis <mdavis@anonymised.com> wrote:

Andrea, I just tried this out, but I think I hit a few bumps:

  • Is the WPS Request Builder supposed to respect the blacklist? Didn’t seem to be working for me

Did you press “save” on the wps admin page? The process selection pages are secondary ones, like the SQL
view ones, you have to save in the main one for the changes to be applied.
I tested it out before sending the first mail, it worked for me

Right, I realized this later on. It does work for me now.

Aside: Is there an important distinction between “Submit” and “Save”? Both seem to be used in the UI in different contexts.

Martin Davis
OpenGeo - http://opengeo.org
Expert service straight from the developers.

On Mon, Aug 13, 2012 at 11:18 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Aug 13, 2012 at 11:45 PM, Martin Davis <mdavis@anonymised.com> wrote:

The wildcarding should make this easy. The default configuration would have “*” in the whitelist, and nothing in the blacklist. This makes all new processes visible by default.

If a more restrictive whitelist was in place, and new processes were being added in a given namespace “ns”, then the admin simply has to add “ns:*” to the whitelist.

I don’t see anything simple in having to remember by heart prefixes and process names, in particular
I find very obnoxious the “limited srs lilst” thing in wms page exactly because it forces one to have
to remember by heart the list of EPSG codes one wants to activate.
I guess some have a much better memory than me, but if the only UI available would be the one you
say in order to fill that box I’d have to grab the WPS capabilities document and start copying and pasting
the names of the processes I want (out of a XML document, ugh), doing it by heart I’d be sure to
introduce typos and other little errors in the list.

Well, there’s the RequestBuilder drop-down, and we’re working on a Process List page. So there’s a few places to find process names. And presumably this is a fairly infrequent activity, and there won’t be 1000’s of processes. Also, the wildcards allow easily referring to groups of processes.

Regarding the while versus black list, the UI I’ve made is very explicit, you see all processes and you
can check on and off the ones you prefer, what makes it a blacklist is the behavior against unknown
processes, if a new process is not in the list it’s allowed to be exposed.

If having a whitelist approach is deemed to be important we can have a checkbox on top of the
groups table stating “Automatically enable new processes” that the admin can check on and off,
and saving in WPSInfo not only the list of the processes that are to be filtered out, but also the
one of the enabled processes, and the checkbox will deal wiith the rest.

I don’t have a strong feeling about having a whitelist as well - it was a just a potential extension which would be fairly easy to add.

I guess I don’t have a clear picture for what your use case for this functionality. I can imagine a couple of different uses:

a) Expose only a few “safe” processes (say some custom ones developed by a organization, or say all the geometry ones). In this case a whitelist would be easier.
b) Remove a few “unwanted” processes (due to failures, or to avoid overuse of resources). A blacklist would work better for this.

Let’s put it like this: we have a -1 from me on such a UI, and a +1 from you of course.
Let’s hear from others, if textbox approach gets a majority feel free to implement it and scrap mine :slight_smile:

Would be interesting to hear input from other GeoServer devs and admins. Or maybe just implement it and get the feedback later. 8^)

The latter would be rather hideous, a developer overwriting another active developer work just based on his
personal judgement.
That’s not how we work around here, we try to reach consensus instead, even if sometimes it is a painful
one to reach.

I just meant that you’ve already committed your work to trunk, so the default path is to leave it in place, and perhaps to hear real-use feedback from other devs and users. It sounds like you have a pressing need for this functionality, and I don’t know how much time we have available to develop a different approach.

As said, let’s have others take a position as well if they think one solution is better than the other, if
we don’t get any development I’d suggest to keep the implementation we already have.

Cheers

Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


Martin Davis
OpenGeo - http://opengeo.org
Expert service straight from the developers.

On Tue, Aug 14, 2012 at 6:25 PM, Martin Davis <mdavis@anonymised.com> wrote:

On Mon, Aug 13, 2012 at 11:08 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Tue, Aug 14, 2012 at 1:59 AM, Martin Davis <mdavis@anonymised.com> wrote:

Andrea, I just tried this out, but I think I hit a few bumps:

  • Is the WPS Request Builder supposed to respect the blacklist? Didn’t seem to be working for me

Did you press “save” on the wps admin page? The process selection pages are secondary ones, like the SQL
view ones, you have to save in the main one for the changes to be applied.
I tested it out before sending the first mail, it worked for me

Right, I realized this later on. It does work for me now.

Aside: Is there an important distinction between “Submit” and “Save”? Both seem to be used in the UI in different contexts.

Nope, there is not

Cheers
Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


On Tue, Aug 14, 2012 at 6:34 PM, Martin Davis <mdavis@anonymised.com> wrote:

Well, there’s the RequestBuilder drop-down, and we’re working on a Process List page. So there’s a few places to find process names. And presumably this is a fairly infrequent activity, and there won’t be 1000’s of processes. Also, the wildcards allow easily referring to groups of processes.

The drop down is not something one can copy and paste, at least not on Linux, I want a typo-less approach.
What would a “Process list” page be? What would its functionality be?

If having a whitelist approach is deemed to be important we can have a checkbox on top of the

groups table stating “Automatically enable new processes” that the admin can check on and off,
and saving in WPSInfo not only the list of the processes that are to be filtered out, but also the
one of the enabled processes, and the checkbox will deal wiith the rest.

I don’t have a strong feeling about having a whitelist as well - it was a just a potential extension which would be fairly easy to add.

I guess I don’t have a clear picture for what your use case for this functionality. I can imagine a couple of different uses:

a) Expose only a few “safe” processes (say some custom ones developed by a organization, or say all the geometry ones). In this case a whitelist would be easier.
b) Remove a few “unwanted” processes (due to failures, or to avoid overuse of resources). A blacklist would work better for this.

As said above, the checkbox approach is both black and white list when it comes to existing processes,
the implementation difference comes up only as you are adding new processes dynamically with scripting.
Or do you have in mind other cases that I’m not thinking about.

Cheers
Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


On Mon, Aug 13, 2012 at 11:08 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

  • The GeoServerProcessors applyFilters method gets called a LOT - e.g. numerous times during constructing a single RequestBuilder page. It seems like if there is a blacklist in place, then it will be evaluated every time this is called, with the creation of a new SelectingProcessFactory each time? Will this be an issue for performance?

The RequestBuilder page is a non use case performance wise, it’s not a proper WPS client and it only gets used by umans.
What could be worrysome is the code path of the OGC requests, that may end up handling hundred of requests
by second, however processes are normally quite time consuming compared to the average OGC request,
so I’m not that worried there either.

Introducing caching in the current design may not be that easy:

  • the factories can be dynamic in the processes they do return
  • the list of filtered processes can change over time
  • the filter themselves can wrap the original factories and introduce
    dynamic behavior based on the user authenticated in the current
    request thread

Yes, I can see that the flexibility of the design may make it difficult to accomodate other goals (such as performance). Looking at the primary requirement of controlling process visibility, there doesn’t seem to be anything about it that inherently precludes caching for performance. Is there a way of implementing this which would allow caching to be added more easily?

That said, I believe it’s possible to create a cache of SelectingProcessFactory
based on the original class (derived by calling getInnermostDelegate on
DelegatingProcessFactory), and then listen to WPS re-configuration events
and drop the cache to make it respect the new rules.
Want to have a crack at it?

Sounds good. Afraid I don’t have time to contribute code for this right now. Hopefully performance won’t be a bottleneck if caching can’t be implemented.

Martin Davis
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Nice - the best of both worlds!

No surprise, I like the idea of persisting to a simple rule file. Various reasons, a primary one being that it follows the design of the current security subsystem.

And if there’s no resources to develop the UI right now, the file can easily be used on its own directly. 8^)

On Tue, Aug 14, 2012 at 7:21 AM, David Winslow <dwinslow@anonymised.com> wrote:

It seems Martin is more interested in power (the ability to white- and black-list individual processes while still being able to have blanket/wildcard rules) and Andrea is more intrested in accessibility (via a point-and-click interface.) As I understand it the current UI defaults to publishing a rule and allows the admin to blacklist blocks of process by factory (and not namespace.)

Instead of switching to a fully text-based interface, perhaps we could make the GUI more sophisticated without sacrificing too much accessibility?

One thought I had was the idea of a more generic “rule” concept - instead of a rule consisting of the pairing of a namespace and a allow/disallow flag, maybe rules could also include a “mode” indicating whether they apply to a namespace or an individual process. A dropdown or auto-completing text field could assist the user in filling out the namespace/process field. I attached a little mockup I made of what that might look like (with a “preview” thrown in at the bottom showing blocked and allowed processes explicitly.)

Or we could have expanders in the rule editing control - maybe each namespace has a little “+” by it to allow making exceptions (so for a blocked namespace, expanding it allows you to choose individual processes to allow, or the other way around for allowed namespaces.)

Depending on how it’s done, the GUI could even persist to a plain text file that MapServer-style admins could hack on :slight_smile:


David Winslow
OpenGeo - http://opengeo.org/

On Tue, Aug 14, 2012 at 2:18 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Aug 13, 2012 at 11:45 PM, Martin Davis <mdavis@anonymised.com> wrote:

The wildcarding should make this easy. The default configuration would have “*” in the whitelist, and nothing in the blacklist. This makes all new processes visible by default.

If a more restrictive whitelist was in place, and new processes were being added in a given namespace “ns”, then the admin simply has to add “ns:*” to the whitelist.

I don’t see anything simple in having to remember by heart prefixes and process names, in particular
I find very obnoxious the “limited srs lilst” thing in wms page exactly because it forces one to have
to remember by heart the list of EPSG codes one wants to activate.
I guess some have a much better memory than me, but if the only UI available would be the one you
say in order to fill that box I’d have to grab the WPS capabilities document and start copying and pasting
the names of the processes I want (out of a XML document, ugh), doing it by heart I’d be sure to
introduce typos and other little errors in the list.

Regarding the while versus black list, the UI I’ve made is very explicit, you see all processes and you
can check on and off the ones you prefer, what makes it a blacklist is the behavior against unknown
processes, if a new process is not in the list it’s allowed to be exposed.

If having a whitelist approach is deemed to be important we can have a checkbox on top of the
groups table stating “Automatically enable new processes” that the admin can check on and off,
and saving in WPSInfo not only the list of the processes that are to be filtered out, but also the
one of the enabled processes, and the checkbox will deal wiith the rest.

Let’s put it like this: we have a -1 from me on such a UI, and a +1 from you of course.
Let’s hear from others, if textbox approach gets a majority feel free to implement it and scrap mine :slight_smile:

Would be interesting to hear input from other GeoServer devs and admins. Or maybe just implement it and get the feedback later. 8^)

The latter would be rather hideous, a developer overwriting another active developer work just based on his
personal judgement.
That’s not how we work around here, we try to reach consensus instead, even if sometimes it is a painful
one to reach.

As said, let’s have others take a position as well if they think one solution is better than the other, if
we don’t get any development I’d suggest to keep the implementation we already have.

Cheers

Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it



Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Martin Davis
OpenGeo - http://opengeo.org
Expert service straight from the developers.

On Tue, Aug 14, 2012 at 8:42 PM, Martin Davis <mdavis@anonymised.com> wrote:

Nice - the best of both worlds!

No surprise, I like the idea of persisting to a simple rule file. Various reasons, a primary one being that it follows the design of the current security subsystem.

And if there’s no resources to develop the UI right now, the file can easily be used on its own directly. 8^)

Sigh, I actually have problems with this as well.

Yes, the old security subsystem was created based on simple property files. The current
one is based on a set of xml ones instead, have a look.

The reason for property files was that there was no money to write a UI back at the time.
And there are still some modules using a simple property or XML file.

The main problem with these is that they play completely outside of the GeoServer configuration
subsystem, in particular:

  • they have no REST counterpart
  • they don’t respond to reload/reset events
  • they make people use the one and only way of configuring GeoServer we repeated for years
    and years is not supported: direct editing of the configuration files on disk
    (GeoServer is not Mapserver, remember?)

The way I’ve just setup the filters instead sticks them into the WPSInfo, which means they
should be editable using the new service configuration subsystem that Juan rolled which…
uh, still does not have the rest resources for WPS… whatever, it should be easy to add.

What I’m trying to say is that the GeoServer configuration should be playing as a single
whole instead of being scattered between files you should not edit and others that
you are forced into editing, as a result it should all be XML.

I can understand adding a extra minor plugin, but this is a major service that I hope
to be able to have as part of core by 2.3.0, so it should try hard to play by the same
rules as the other major services.

Cheers
Andrea

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it