[Geoserver-devel] GSIP 191 - New extension point in WMS rendering to enrich style symbolizers

Hi Project Steering Committee and Community,

I would like to start the voting stage for a new proposal titled “New extension point in WMS rendering to enrich style symbolizers” with assigned code GSIP 191.

This is a proposal for adding an extension point for processing and enriching style symbolizers in base to the feature being rendered.

The technical detail in the proposal document:
https://github.com/geoserver/geoserver/wiki/GSIP-191

Naturally, I’ll be glad to answer any question or discuss about the proposal and implementation details.

Thanks for your vote :slight_smile:

Regards,

Fernando Mino

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

==

Fernando Mino

Software Engineer

GeoSolutions S.A.S.

Via di Montramito 3/A

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

http://www.geo-solutions.it

http://twitter.com/geosolutions_it


Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Happy to review the proposal, but the title is very generic - and “processing and enriching” has me curious?

Reading the proposal ends up defining the following callback:

interface SymbolizersCallback {

/** Can modify the list of symbolizers to be painted */

List apply(Feature feature, List symbolizers);

}

Are you sure you are working at the right level of abstraction, the activity you are doing above is close in spirit to process stylers in the streaming renderer?

I am trying to figure out higher level what you are trying to accomplish? Are you wishing to make up styles on the fly … if so the result probably cannot be written out out via SLD, and may as well be your own rendering engine for a layer. The GeoTools rendering model has DirectLayer for this kind of case, used for things like map decorations, or mil2525b.

I think this proposal needs some discussion before being ready to vote.

···

Jody Garnett

Quick question, the proposal indicates you are going to make use of this callback “before every code point where the StreamingRenderer Geotools class is instanced”…
But this callback is on a feature by feature basis.

Do you mean to make a geotools proposal to define this callback, and then make use of the new API in geoserver?

···


Jody Garnett

Hi Jody, thanks for start the discuss about this proposal.

The goal of this extension point is to give the possibility to enrich/decorate current feature styling. The feature defined style can be maintained as is, and it will be possible to add, for example, a surrounded circle marking the feature.

This extension point will make viable an upcoming dynamic features highlighting community module. On this implementation, we’ll be able to add highlight marks to any-layer current features style based on request vendor parameter; so the module will highlight some features and leave the remaining others as default.

Naturally, if other Geotools/Geoserver extension point already allows us to make this goal, it would be awesome to save unneeded development. Do you have some lights about how to to this with existing GT/GS extensions?

Thanks for your help :slight_smile:

Regards,

Fernando Mino

···

Jody Garnett

About the quick question, since I’m planing to extend StreamingRenderer class in Geoserver WMS, I only need to make the processSymbolizers method protected (it is currently private). Do I need to make a Geotools proposal for this change?

Is this plan good enough, or it is required to extend the Geotools API for handle by-feature interceptors(preprocessing objects), calling them inside the processSymbolizers code?

Thanks!

Regards,

Fernando Mino

···


Jody Garnett

In terms of design it best to do the work in the project responsible for the abstractions, if not you risk getting broken easily by upstream changes. It is also nice for the maintainers to have all the cards on the table in terms of how the class is used, so they can make an informed decision on making a method protected, or making a new method for your call back to use. Andrea is largely responsible for this class, if I was maintaining it I would want to define strictly any callbacks in StreamingRenderer so I could understand what it is possible to optimize, or when I need to go slow to let a callback happen.

We should move the geotools discussion to that list, just in terms of what you want to accomplish it looks like you may be at the wrong spot (working with SE symbolizers rather then the java 2d based style).

···


Jody Garnett

Thanks for the context, that helps:

  • In SLD highlighting is done as a distinct layer, allowing a highlight to be dawn in a consistent fashion and managed in the z-order (incase content from several layers is considered selected and needs to be highlighted). The CQL_Filter is very handy for this case, and indeed was added for this use.

  • In uDig (a desktop environment) the equivalent of a DirectLayer was used for this purpose; the result was powerful ending up similar to labels where content from several layers could be floated onto of the map to show selection.

Given you requirements (a circle) having a specific layer seems fine? But perhaps I do not see the entire picture yet.

···


Jody Garnett

Hi Jody,

Thanks for your nice hints, indeed those are interesting solutions, but I think they are unsuitable for the use case I would like to implement (I can be wrong, if I am please help me to understand how to use the existing tooling to achieve this task).

Probably, as you say, I failed on drawing the entire picture, so I’ll put all the functionality I need from this extension point from the upcoming community module perspective:

  • Dynamic features highlight will allow the client to send which features to mark in the same layer/WMS request.
  • This highlight mark will come in form of a different color shape surrounding the marked feature on the output image. For example a red circle surrounding a feature icon.
  • This mark draw will be added to the current feature style. It will not replace the defined style for the layer/feature.
  • For defining which features to mark, a CQL_FILTER will be used. This filter is not the same as the WMS request filter.
  • It will allow to send several hightlight definitions with different filters, colors and shapes.
  • Highlight will be requested by WMS GetMap request, so for every WMS GetMap request a different highlight setup can be retrieved.
  • For achieving this, it will use a new vendor parameter, called ‘HIGHLIGHT’.

The HIGHLIGHT vendor parameter will receive, comma separated parameters and multiple commands semicolon separated:

  • layer name
  • CQL filter
  • highlight color
  • WKT shape identifier
  • stroke size (optional)
  • margin size (optional)

Example:
HIGHLIGHT=
st:stations,id IN (100,101,102),#FF0000,circle,2,1;
st:stations,id IN (102,103,104),#00FF00,square,2,1;
st:stations,id IN (104,105,106),#0000FF,triangle,2,1;

I attached an highlight output example image.

Regards,

Fernando Mino

(attachments)

highlight-example.png

···


Jody Garnett

I’m not clear on why this can’t just be another WMS request - pass in a new style or SLD and a CQL_FILTER and away you go. Since there is no point in highlighting more than a handful of points on any map it should be fast and easy (assuming the ID or what ever is indexed).

Am I missing something?

Ian

···

Ian Turton

Hi Ian,

The approach with dynamic highlighting is that the client can dynamically request unlimited mark combinations of color, shape, marked features, and obtain them in the same WMS GetMap picture without defining new styles for every highlight mark layer/color/shape.

And the other key is: when for example 2 highlights marks with same shape are applied to a same feature, the marks will be drawn with relative-different size, so both mark colors will be visible on the output image.

Regards,

Fernando Mino

···

Ian Turton

My two cents to this.

@Ian

Your approach is correct but it does not work on moving data because we can certainly request the same layer twice with different style and CQL but the underlying database request will be redundant and might end up fetching same feature at different locations. So this new vendor option Highlight will ensure that a feature retrieved once can be rendered multiple times.

I hope I was able to make a strong case :slight_smile:

regards,
Imran

···

I.R

On Wed, 6 May 2020 at 15:52, Imran Rajjad <rajjad@anonymised.com> wrote:

My two cents to this.

@Ian

Your approach is correct but it does not work on moving data because we can certainly request the same layer twice with different style and CQL but the underlying database request will be redundant and might end up fetching same feature at different locations. So this new vendor option Highlight will ensure that a feature retrieved once can be rendered multiple times.

If your feature is moving that fast then I would use a WFS and highlight it directly in the client - Alternatively exclude the highlighted feature from the bottom layer and just draw it in the highlight layer.

Ian

On Wed, 6 May 2020 at 14:54, <fernando.mino@anonymised.com> wrote:

Hi Ian,

The approach with dynamic highlighting is that the client can dynamically request unlimited mark combinations of color, shape, marked features, and obtain them in the same WMS GetMap picture without defining new styles for every highlight mark layer/color/shape.

And the other key is: when for example 2 highlights marks with same shape are applied to a same feature, the marks will be drawn with relative-different size, so both mark colors will be visible on the output image.

This still seems like a big change for a relatively simple issue that has been handled for many years without it. I can see that you may need to define a small number of SLDs but that has to be quicker and easier than adding a whole new extension point.

Ian

This seems a bit like you are just defining a new highlight layer, poorly using different parameters? Like you your proposed vendor options are a “mini” layer definition:

  • layer name
  • CQL filter
  • highlight color
  • WKT shape identifier
  • stroke size (optional)
  • margin size (optional)

You can do this already with a GetMap request and an SLD, why make life (and our codebase) more complicated?

···


Jody Garnett

I haven’t tested it in eons, but it looks to me like what you’re after can be done by WMS GetMap using HTTP POST and its corresponding XML payload?

···

Gabriel Roldán

Dear all,

I’m working with @Fernando on this one, and thank you for all the feedback so far!

So first of all, @Jody is right, two proposals should be made one in GeoTools and one in GeoServer.

That said, I would like to recentre the discussion about the extension point itself rather than on one of the particular use cases we have, where not all details were provided:

  • A significant number of features will need to be highlighted (up to 20000).

  • The highlight will be dynamic per feature based on several criteria’s:

  • A feature that matches N highlighting conditions should be highlighted N times.

  • Fast moving objects, querying the system twice may result in misalignments between the features layer and the highlighted layer.

What we are proposing here is an extension point on the rendering engine (both GeoTools and GeoServer) that will allow extensions (or even GeoServer itself) to create symbolizers on the fly and apply them based on some custom logic without requiring a static SLD definition for them.

The key notion here it’s the ability to allow extensions to generate during runtime symbolizers using a custom logic that can be quite complex and not easy, not desirable or even impossible to express in SLD.

We are going to refactor the GeoServer proposal to focus on the main point and create a proposal for the callback on GeoTools.

Kind regards,

Nuno Oliveira

···

Regards,
Nuno Oliveira

GeoServer Professional Services from the experts!
Visit http://goo.gl/it488V for more information.

Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


Con riferimento alla normativa sul trattamento dei dati
personali (Reg. UE 2016/679 - Regolamento generale sulla
protezione dei dati “GDPR”), si precisa che ogni
circostanza inerente alla presente email (il suo contenuto,
gli eventuali allegati, etc.) è un dato la cui conoscenza
è riservata al/i solo/i destinatario/i indicati dallo
scrivente. Se il messaggio Le è giunto per errore, è
tenuta/o a cancellarlo, ogni altra operazione è illecita.
Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to
which it is addressed and may contain information that
is privileged, confidential or otherwise protected from
disclosure. We remind that - as provided by European
Regulation 2016/679 “GDPR” - copying, dissemination or
use of this e-mail or the information herein by anyone
other than the intended recipient is prohibited. If you
have received this email by mistake, please notify
us immediately by telephone or e-mail.

Nuno we are struggling a bit with what you propose because you are wishing to make up symbolizers on a feature by feature basis. My understanding is streaming renderer goes to a lot of work to review what is intended to be a static style in order to break the rendering task up into buffers, z-orders, sorting, queries, etc…
I am concerned that streaming renderer is very complicated and that by changing the symbolizers no the fly you will either upset performance (never a good idea) or make the code even harder to maintain (when you try and restore performance.

For work that is not desirable to express in SLD we do have an “out” in the form of GeoTools Direct Layer. All the same same functionality available to streaming renderer (converting geometry to Java 2D shapes) is available for you to use in your own implementation. You could even try subclassing StreamingRenderer, or forking it, as was done with previous shapefile streaming renderer.

Still there is not quite enough for me to go on yet, I look forward to your geotools proposal and a chance to listen to what you have in mind.

···


Jody Garnett

Nuno we are struggling a bit with what you propose because you are wishing to make up symbolizers on a feature by feature basis. My understanding is streaming renderer goes to a lot of work to review what is intended to be a static style in order to break the rendering task up into buffers, z-orders, sorting, queries, etc…

Yes. Most of that is done at the feature type level though (besides attribute selection, see below).
Adding new symbolizers on a feature by feature basis is not going to have an effect on that logic.
If the extension point allowed to modify feature types and rules, then yes, it would be harder (whether it would be too much, or not, that’s another story).

I am concerned that streaming renderer is very complicated and that by changing the symbolizers no the fly you will either upset performance (never a good idea) or make the code even harder to maintain (when you try and restore performance.

Neither seems to be true to me… most of the optimizations happen at the higher levels.
If the extension point is called at the beginning of processSymbolizers, we are at a point where
the code is really just executing painting instructions for that one feature.

The one downside that there might be, is that the renderer will give the extension point a feature whose attributes have been reduced to the ones
explicitly used in the style, it will have to work with what it has there. Thinking out loud, it could be extended a bit more to have a voice
regarding the attribute selection, giving it the ability to claim for extra attributes needed to do its work.

For work that is not desirable to express in SLD we do have an “out” in the form of GeoTools Direct Layer. All the same same functionality available to streaming renderer (converting geometry to Java 2D shapes) is available for you to use in your own implementation. You could even try subclassing StreamingRenderer, or forking it, as was done with previous shapefile streaming renderer.

I have stayed out of this discussion so far… In my opinion, the single focus on dynamic highlight muddied the waters.
The way I see it, an extension point like this one allows to do something that would be hard or impossible to do with SLD and its static structure limits, including:

  • Complex calculation that could be hard, or very inefficient, to run as OGC Expressions. Context, I had to do several rounds of optimizations in the GeoTools expression evaluation engine to make the OSM styles perform, and yet, it’s still very slow, has an allocation rate that’s just murder due to the “everything is an object” paradigm and widespread usage of converters. Will never be able to compete with a direct java implementation of the same logic, by design.
  • Stateful evaluation, in particular, keeping track of interesting features and so something different based on their re-appearance or overlap with other layers. For example, in marine cartography one needs to change symbology based on what’s overlapping in previously painted layers… An SLD extension doing something like this would have to change the language in a significant way, I believe.
  • Out of band information. The SLD painting model is pretty simple, can only play with the layers that are included in the map being painted, one at a time. A programmable extension point would allow to reach out of these limits, and perform decisions based on information present outside of the current map layers.
    Rendering transformations could also be used to achieve some of the above goals, but they would have to act without knowing how the feature is going to be depicted, while this extension point gets the symbolizers, and can take decisions based on them too.

Cheers
Andrea

···

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Thanks PMC and community members for your feedback. In base to it I splitted the proposal into a Geotools and a GeoServer proposals.

I added some important suggestions from Andrea feedback like the required attributes declaration and a by-layer render buffer declaration on the extension point, remaining the core functionality as the former one, but now making the extension interface and registering part of Geotools StreamingRenderer class.

Is this proposals organization and content better?

Naturally I’ll be glad to receive your feedback or answer any doubt.

As note: Naming is hard, we thought of calling it SymbolizersPreProcessor, do you have any suggestion for a better name?

Geotools proposal draft:

Thanks :slight_smile:

Regards,

Fernando Mino

···

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.