[Geoserver-devel] Adding CQL filtering at the layer level configuration

GeoServer GeoServer Developer
1

Hi,
in this mail I’m proposing we resurrect an old functinality we had at the 1.7.x times,
filtering layers by a user specified filter.

The FeatureTypeInfo filter field is indeed still there:
https://github.com/geoserver/geoserver/blob/master/src/main/src/main/java/org/geoserver/catalog/FeatureTypeInfo.java#L61

but the code around it is kind of lacking, and need some massaging and deep testing to make
sure it actually works, since nothing in that area has been touched in several years.

GUI wise we need a way to edit the filters, which I’m proposing is going to be a simple text field for the moment,
that will show up below the attribute list, and which will validate checking both syntax and proper usage of
existing attributes in the layer.

Persistence wise, we need to have XStream store the Filter (right now there are no converters for them, which
would make for a non useful storage), which I’m proposing to store as ECQL too.

We’ll then have to verify that the filtering machinery works at the GeoServerFeatureSource level, and that
it mixes well with user, style and security filters.

The impetus to work on this originates from simplifying and at the same time generalizing the SORL store
usage (see related mail in GeoTools devel),
dropping the attribute based layer mapping, and replacing with a more general CQL filter that
will also allow the same document to show up in multiple layers, but the functionality is general and will
be useful in other cases too.

Cheers
Andrea

···

==
Meet us at the INSPIRE Conference in Lisbon 25-29 May 2015! Visit http://goo.gl/WHKDXT for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

2

Andrea,

This is a technical comment/question rather than a note about the main proposal. (The proposal overall seems sensible to me.)

As you work with and store the ECQL filter, are you going to need ECQL.toFilter/ECQL.toCQL to be inverses? I’ve seen a few cases where that doesn’t work out completely. (And I’ve tried to help submit patches as I learn about them.)

Anyhow, maybe that’s not necessary; figured I’d mention it.

Cheers,

Jim

···

On 05/20/2015 05:43 AM, Andrea Aime wrote:

Hi,
in this mail I’m proposing we resurrect an old functinality we had at the 1.7.x times,
filtering layers by a user specified filter.

The FeatureTypeInfo filter field is indeed still there:
https://github.com/geoserver/geoserver/blob/master/src/main/src/main/java/org/geoserver/catalog/FeatureTypeInfo.java#L61

but the code around it is kind of lacking, and need some massaging and deep testing to make
sure it actually works, since nothing in that area has been touched in several years.

GUI wise we need a way to edit the filters, which I’m proposing is going to be a simple text field for the moment,
that will show up below the attribute list, and which will validate checking both syntax and proper usage of
existing attributes in the layer.

Persistence wise, we need to have XStream store the Filter (right now there are no converters for them, which
would make for a non useful storage), which I’m proposing to store as ECQL too.

We’ll then have to verify that the filtering machinery works at the GeoServerFeatureSource level, and that
it mixes well with user, style and security filters.

The impetus to work on this originates from simplifying and at the same time generalizing the SORL store
usage (see related mail in GeoTools devel),
dropping the attribute based layer mapping, and replacing with a more general CQL filter that
will also allow the same document to show up in multiple layers, but the functionality is general and will
be useful in other cases too.

Cheers
Andrea

–

==
Meet us at the INSPIRE Conference in Lisbon 25-29 May 2015! Visit http://goo.gl/WHKDXT for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
[http://ad.doubleclick.net/ddm/clk/290420510;117567292;y](http://ad.doubleclick.net/ddm/clk/290420510;117567292;y)

_______________________________________________
Geoserver-devel mailing list
[Geoserver-devel@lists.sourceforge.net](mailto:Geoserver-devel@lists.sourceforge.net)
[https://lists.sourceforge.net/lists/listinfo/geoserver-devel](https://lists.sourceforge.net/lists/listinfo/geoserver-devel)
3

On Wed, May 20, 2015 at 7:56 PM, Jim Hughes <jnh5y@anonymised.com> wrote:

Andrea,

This is a technical comment/question rather than a note about the main
proposal. (The proposal overall seems sensible to me.)

As you work with and store the ECQL filter, are you going to need
ECQL.toFilter/ECQL.toCQL to be inverses? I've seen a few cases where that
doesn't work out completely. (And I've tried to help submit patches as I
learn about them.)

I'm aware, but I guess we'll cross that river when necessary. I don't think
it's a big deal if the syntax changes, provided
the semantic of the filter remains the same. Or not?

Cheers
Andrea

--

Meet us at the INSPIRE Conference in Lisbon 25-29 May 2015! Visit
http://goo.gl/WHKDXT for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

4

The times I’ve run into it, ECQL.toCQL has produced a string which cannot be parsed by ECQL.toFilter.

If you carried around the original input string, you’d be set. For us, we’ve played rewriting filters and that’s when things have gotten exciting.

Cheers,

JIm

···

On 05/20/2015 02:01 PM, Andrea Aime wrote:

On Wed, May 20, 2015 at 7:56 PM, Jim Hughes <jnh5y@anonymised.com> wrote:

Andrea,

This is a technical comment/question rather than a note about the main proposal. (The proposal overall seems sensible to me.)

As you work with and store the ECQL filter, are you going to need ECQL.toFilter/ECQL.toCQL to be inverses? I’ve seen a few cases where that doesn’t work out completely. (And I’ve tried to help submit patches as I learn about them.)

I’m aware, but I guess we’ll cross that river when necessary. I don’t think it’s a big deal if the syntax changes, provided
the semantic of the filter remains the same. Or not?

Cheers
Andrea

–

==
Meet us at the INSPIRE Conference in Lisbon 25-29 May 2015! Visit http://goo.gl/WHKDXT for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

5

On Wed, May 20, 2015 at 8:14 PM, Jim Hughes <jnh5y@anonymised.com> wrote:

The times I've run into it, ECQL.toCQL has produced a string which cannot
be parsed by ECQL.toFilter.

If you carried around the original input string, you'd be set. For us,
we've played rewriting filters and that's when things have gotten exciting.

Hi Jim,
in the end we've followed your advice and kept the CQL unparsed, to
generate the Filter representation
on the fly.

It's part of this pull request:
https://github.com/geoserver/geoserver/pull/1128

In particular, this commit:
https://github.com/aaime/geoserver/commit/3654db29f94360a378693868e0c4fd5d62a21f2f

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------