[Geoserver-devel] Again on filters (this time for good :-) )

Hi all,
I'm overhauling the filter parsing code so that WFS GetFeature
and WMS GetMap share the same code. I'll be adding CQL filters
parsing too.
In this process I need two new parameters:
* FILTER_DELIMITER, as proposed by Saul, to allow building filter lists
   that do use separators other than () (this is especially important
   for CQL filters, since ( ) are mathematical parenthesis there
   (see http://docs.codehaus.org/display/GEOTOOLS/CQL+Samples)
   Example: ...&FILTER_DELIMITER={}&...
* CQL_FILTER, that will include a list of CQL filters. Having an
   extra parameter eases parsing, but I could keep using FILTER as well,
   and try to parse with CQL should OGC parsing go bad.
   A separate parameter seems clearer to me, but that's just me.

Suggestions, questions, altenative proposals?
Cheers
Andrea

Andrea Aime wrote:

Hi all,
I'm overhauling the filter parsing code so that WFS GetFeature
and WMS GetMap share the same code. I'll be adding CQL filters
parsing too.
In this process I need two new parameters:
* FILTER_DELIMITER, as proposed by Saul, to allow building filter lists
   that do use separators other than () (this is especially important
   for CQL filters, since ( ) are mathematical parenthesis there
   (see http://docs.codehaus.org/display/GEOTOOLS/CQL+Samples)
   Example: ...&FILTER_DELIMITER={}&...
* CQL_FILTER, that will include a list of CQL filters. Having an
   extra parameter eases parsing, but I could keep using FILTER as well,
   and try to parse with CQL should OGC parsing go bad.
   A separate parameter seems clearer to me, but that's just me.

I think CQL_FILTER sounds fine to me. You're adding it to WFS as well, right? I think it'd be great for web developers to not have to construct the whole XML filter, but instead to just say much closer to what they mean.

We should probably submit both of these as Change Requests to the WFS spec.

Chris

Suggestions, questions, altenative proposals?
Cheers
Andrea

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:1003,45d07c22225301510810322!

--
Chris Holmes
The Open Planning Project
http://topp.openplans.org

Chris Holmes ha scritto:

Andrea Aime wrote:

Hi all,
I'm overhauling the filter parsing code so that WFS GetFeature
and WMS GetMap share the same code. I'll be adding CQL filters
parsing too.
In this process I need two new parameters:
* FILTER_DELIMITER, as proposed by Saul, to allow building filter lists
   that do use separators other than () (this is especially important
   for CQL filters, since ( ) are mathematical parenthesis there
   (see http://docs.codehaus.org/display/GEOTOOLS/CQL+Samples)
   Example: ...&FILTER_DELIMITER={}&...
* CQL_FILTER, that will include a list of CQL filters. Having an
   extra parameter eases parsing, but I could keep using FILTER as well,
   and try to parse with CQL should OGC parsing go bad.
   A separate parameter seems clearer to me, but that's just me.

I think CQL_FILTER sounds fine to me. You're adding it to WFS as well, right? I think it'd be great for web developers to not have to construct the whole XML filter, but instead to just say much closer to what they mean.

Yup, the idea was to share common filter parsing code and more or less
same parameters.

We should probably submit both of these as Change Requests to the WFS spec.

That would be a nice move indeed.

Cheers
Andrea

The WFS filter parsing on trunk is not using this code anymore. Its using GTXML. WMS is still using the old stuff, so this is kind of counter to your efforts.

I am more for a separate parameter for cql parsing. Going on a failure case basis isn't really reliable. For things like wfs cite, you have to report back certain failures when misbehaving filters are encountered. Failing and then recovering and trying parse as a cql_filter will result in a different error. Then one is left with the question, well should I just throw the first? What if for some bizarre reason the the filter actually parses the second time, etc...

-Justin

Andrea Aime wrote:

Hi all,
I'm overhauling the filter parsing code so that WFS GetFeature
and WMS GetMap share the same code. I'll be adding CQL filters
parsing too.
In this process I need two new parameters:
* FILTER_DELIMITER, as proposed by Saul, to allow building filter lists
   that do use separators other than () (this is especially important
   for CQL filters, since ( ) are mathematical parenthesis there
   (see http://docs.codehaus.org/display/GEOTOOLS/CQL+Samples)
   Example: ...&FILTER_DELIMITER={}&...
* CQL_FILTER, that will include a list of CQL filters. Having an
   extra parameter eases parsing, but I could keep using FILTER as well,
   and try to parse with CQL should OGC parsing go bad.
   A separate parameter seems clearer to me, but that's just me.

Suggestions, questions, altenative proposals?
Cheers
Andrea

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:1004,45d07c22225331194215290!

--
Justin Deoliveira
jdeolive@anonymised.com
The Open Planning Project
http://topp.openplans.org

Hi Andrea - before you get going can you read the catalog spec (that defines CQL as well as the idea of a query language). You should make sure that the style of "filter parsing" you define would be usable to a catalog service, specifically account for a query language parameter that tells you which parser to use.

QUERY_LANGAGE would subsume the FILTER_DELIMITER parameter
FILTER would be a parameter to parse according to which QUERY language was specified, CQL being the default

Does the WFS1.1 specification have any guidance in this regard Justin?
Cheers,
Jody

Hi all,
I'm overhauling the filter parsing code so that WFS GetFeature
and WMS GetMap share the same code. I'll be adding CQL filters
parsing too.
In this process I need two new parameters:
* FILTER_DELIMITER, as proposed by Saul, to allow building filter lists
   that do use separators other than () (this is especially important
   for CQL filters, since ( ) are mathematical parenthesis there
   (see http://docs.codehaus.org/display/GEOTOOLS/CQL+Samples)
   Example: ...&FILTER_DELIMITER={}&...
* CQL_FILTER, that will include a list of CQL filters. Having an
   extra parameter eases parsing, but I could keep using FILTER as well,
   and try to parse with CQL should OGC parsing go bad.
   A separate parameter seems clearer to me, but that's just me.

Suggestions, questions, altenative proposals?
Cheers
Andrea

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
  

Jody Garnett ha scritto:

Hi Andrea - before you get going can you read the catalog spec (that defines CQL as well as the idea of a query language). You should make sure that the style of "filter parsing" you define would be usable to a catalog service, specifically account for a query language parameter that tells you which parser to use.

QUERY_LANGAGE would subsume the FILTER_DELIMITER parameter
FILTER would be a parameter to parse according to which QUERY language was specified, CQL being the default

CQL cannot be the default, we're talking about WFS too here, OGC should
be the default (we want the same code for both WMS and WFS GET requests).
I don't have time to read the 205 pages catalogue spec, but it seems
to me you're referring to the CONSTRAINTLANGUAGE parameter for the
GetRecords operation? Values are CQL_TEXT and FILTER, how I'm supposed
to handle the separator with that?

I don't know if you remember the issue, parameter parsing is not that
smart, if anyone tries to issue an OGC filter that uses ( or ) inside
a string it will fail, that's why we wanted to add a FILTER_DELIMITER
parameter, so that you could use {} for OGC filters as well.
Oh, another thing, the catalogue stuff does not seem to define a separator, since the filter is just one.

Cheers
Andrea

Justin Deoliveira ha scritto:

The WFS filter parsing on trunk is not using this code anymore. Its using GTXML. WMS is still using the old stuff, so this is kind of counter to your efforts.

I am more for a separate parameter for cql parsing. Going on a failure case basis isn't really reliable. For things like wfs cite, you have to report back certain failures when misbehaving filters are encountered. Failing and then recovering and trying parse as a cql_filter will result in a different error. Then one is left with the question, well should I just throw the first? What if for some bizarre reason the the filter actually parses the second time, etc...

Oh hum, trying to keep everybody happy... say we have a FILTERLANGUAGE
parameter, defaulting to "FILTER", with the other value "CQL_TEXT" (same as catalogue), and we keep FILTER a single parameter interpreted in two
different ways according to the language... would that work for you?
The FILTER_DELIMITER parameter is going to stay, it does not seem
to fit into FILTERLANGUAGE to me (you can always try to change my mind thought).

Cheers
Andrea

Andrea Aime wrote:

Justin Deoliveira ha scritto:

The WFS filter parsing on trunk is not using this code anymore. Its using GTXML. WMS is still using the old stuff, so this is kind of counter to your efforts.

I am more for a separate parameter for cql parsing. Going on a failure case basis isn't really reliable. For things like wfs cite, you have to report back certain failures when misbehaving filters are encountered. Failing and then recovering and trying parse as a cql_filter will result in a different error. Then one is left with the question, well should I just throw the first? What if for some bizarre reason the the filter actually parses the second time, etc...

Oh hum, trying to keep everybody happy... say we have a FILTERLANGUAGE
parameter, defaulting to "FILTER", with the other value "CQL_TEXT" (same as catalogue), and we keep FILTER a single parameter interpreted in two
different ways according to the language... would that work for you?
The FILTER_DELIMITER parameter is going to stay, it does not seem
to fit into FILTERLANGUAGE to me (you can always try to change my mind thought).

Better... but it still involves knowing about another parameter to properly parse the "FILTER" parameter which I am not crazy about... I may be mistaken but for the most part, key-value parameters are usually self contained.

I am also a little bit weary about modifying a parameter which is clearly documented by the spec to be a filter in the traditional sense. I would be more comfortable with "extending" the spec with a new parameter.

Cheers
Andrea

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:1004,45d0b048281207785049143!

--
Justin Deoliveira
jdeolive@anonymised.com
The Open Planning Project
http://topp.openplans.org

Not sure I have time (or want to) change your mind. But I can explain my thought ...
the FILTER KvP is defined by a specification (and we cannot mess with that even
with respect to changing a delimiter) ...

So either we can ..
- add new KvP entries ... CQL_FILTER (for the catalog one) and FILTER2 (for the other kind of delimiter), SFSQL and so on ... but his would get tiring after a while
- use the exact same KvP entries as catalog (CONSTRAINTLANGUAGE and ???) as that approach scales and should be in keeping with where WFS spec is going

You guys have access to the WFS lists; why not check directly and see if there is any direction on these issues...
Jody
Andrea Aime wrote:

Oh hum, trying to keep everybody happy... say we have a FILTERLANGUAGE
parameter, defaulting to "FILTER", with the other value "CQL_TEXT" (same as catalogue), and we keep FILTER a single parameter interpreted in two
different ways according to the language... would that work for you?
The FILTER_DELIMITER parameter is going to stay, it does not seem
to fit into FILTERLANGUAGE to me (you can always try to change my mind thought).

Cheers
Andrea

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
  

Jody Garnett ha scritto:

Not sure I have time (or want to) change your mind. But I can explain my thought ...
the FILTER KvP is defined by a specification (and we cannot mess with that even with respect to changing a delimiter) ...

I don't understand, standard requests would perform just fine, extended
requests may use a different delimiter, since they are not standard anyways. What I'm missing?
The catalogue standard does not define a delimiter, so I may need
the user to pick up one anyways (since even with CQL I have to split
multiple filters before passing each to the parser, or not, Gabriel?)

So either we can ..
- add new KvP entries ... CQL_FILTER (for the catalog one) and FILTER2 (for the other kind of delimiter), SFSQL and so on ... but his would get tiring after a while
- use the exact same KvP entries as catalog (CONSTRAINTLANGUAGE and ???) as that approach scales and should be in keeping with where WFS spec is going

These would be CONTRAINTLANGUAGE and CONSTRAINT. Using these verbatim would scale and would not mess with WFS indeed.
Well, for starters I could add CONSTRAINT to both WMS and WFS, assuming
it's expressed in CQL, and if we add support for other filtering languages, we can then add CONSTRAINTLANGUAGE as well.
I'm not particularly fond of this solution (would have preferred FILTERLANGUAGE and FILTER, with default as OGC_FILTER) but since both
of you are agaisnt it...

You guys have access to the WFS lists; why not check directly and see if there is any direction on these issues...

Bleah, I hate doing this under release, but let's do it anyways.
Cheers
Andrea

Maybe I am missing; what do you mean by delimiter?

My point was basically if it is an extension it should be a new KvP. If the extension is about filter; perhaps our extension could *be* the same KvP from the catalog query (so we have less email discussion and documentation to think through).

Cheers,
Jody

Andrea Aime wrote:

I don't understand, standard requests would perform just fine, extended
requests may use a different delimiter, since they are not standard anyways. What I'm missing?
The catalogue standard does not define a delimiter, so I may need
the user to pick up one anyways (since even with CQL I have to split
multiple filters before passing each to the parser, or not, Gabriel?)

Jody

Jody Garnett ha scritto:

Maybe I am missing; what do you mean by delimiter?

WFS requires one filter per layer:

FILTER=(<Filter>...</Filter>)(...)

( and ) are the WFS standard delimiters. Our parsing breaks if any
of these is used inside the filter, such as in a Literal.
For proper parsing I would need something that stops when the
first filter ends, allows me to process the delimiters, and then
start again parsing the filter, and so on.

With CQL the issue is more or less the same, I obviously cannot
use ( and ) as default delimiters since they are used in many
expressions, so I'll probably use { and } instead, but the
issue is the same, what if a literal uses them?
You need a way to specify another delimiter, a char the user knows will
never be used by his queries.

Cheers
Andrea

Andrea Aime wrote:

Jody Garnett ha scritto:

Maybe I am missing; what do you mean by delimiter?

WFS requires one filter per layer:

FILTER=(<Filter>...</Filter>)(...)

( and ) are the WFS standard delimiters. Our parsing breaks if any
of these is used inside the filter, such as in a Literal.

Can we fix our parsing?

For proper parsing I would need something that stops when the
first filter ends, allows me to process the delimiters, and then
start again parsing the filter, and so on.

Oh okay; Justin does your GTXML streaming parser do this? I think it does since we can read the "next" feature...

With CQL the issue is more or less the same, I obviously cannot
use ( and ) as default delimiters since they are used in many
expressions, so I'll probably use { and } instead, but the
issue is the same, what if a literal uses them?

Is this not really a case of not paying attention to nesting? ie if we paid attention
to nesting (and quotes) then ( and ) would be fine?

You need a way to specify another delimiter, a char the user knows will
never be used by his queries.

This seems a bit rough, what does the specification say?
Jody

Jody Garnett wrote:

Andrea Aime wrote:

Jody Garnett ha scritto:

Maybe I am missing; what do you mean by delimiter?

WFS requires one filter per layer:

FILTER=(<Filter>...</Filter>)(...)

( and ) are the WFS standard delimiters. Our parsing breaks if any
of these is used inside the filter, such as in a Literal.

Can we fix our parsing?

For proper parsing I would need something that stops when the
first filter ends, allows me to process the delimiters, and then
start again parsing the filter, and so on.

Oh okay; Justin does your GTXML streaming parser do this? I think it does since we can read the "next" feature...

No since this things are not a single document, they are multipel documents divided by a delimiter.

With CQL the issue is more or less the same, I obviously cannot
use ( and ) as default delimiters since they are used in many
expressions, so I'll probably use { and } instead, but the
issue is the same, what if a literal uses them?

Is this not really a case of not paying attention to nesting? ie if we paid attention
to nesting (and quotes) then ( and ) would be fine?

You need a way to specify another delimiter, a char the user knows will
never be used by his queries.

This seems a bit rough, what does the specification say?
Jody

!DSPAM:1004,45d0cd9c309222095110867!

--
Justin Deoliveira
jdeolive@anonymised.com
The Open Planning Project
http://topp.openplans.org

Jody Garnett ha scritto:

Andrea Aime wrote:

Jody Garnett ha scritto:

Maybe I am missing; what do you mean by delimiter?

WFS requires one filter per layer:

FILTER=(<Filter>...</Filter>)(...)

( and ) are the WFS standard delimiters. Our parsing breaks if any
of these is used inside the filter, such as in a Literal.

Can we fix our parsing?

Not after an RC is out, no. I can delay the issue thought and live
like I did not know about the problem.

With CQL the issue is more or less the same, I obviously cannot
use ( and ) as default delimiters since they are used in many
expressions, so I'll probably use { and } instead, but the
issue is the same, what if a literal uses them?

Is this not really a case of not paying attention to nesting? ie if we paid attention
to nesting (and quotes) then ( and ) would be fine?

Fine and confusing, try and read three filters sticked toghether that do
use ( and ) inside them too and tell me which is which.

You need a way to specify another delimiter, a char the user knows will
never be used by his queries.

This seems a bit rough, what does the specification say?

Nothing, ( and ) are the only delimiters in WFS, and catalogues does not
provide one in my cursory look at it.

Cheers
Andrea

Jody Garnett ha scritto:

Andrea Aime wrote:

Jody Garnett ha scritto:

Maybe I am missing; what do you mean by delimiter?

WFS requires one filter per layer:

FILTER=(<Filter>...</Filter>)(...)

( and ) are the WFS standard delimiters. Our parsing breaks if any
of these is used inside the filter, such as in a Literal.

Can we fix our parsing?

Jody, the issue here is that OGC decided to mix two parsing
tecniques in the same spec. It would have been a lot easier
if the FILTER element did not have separators at all, and used
a structure like:

<FILTERS><FILTER>...</FILTER><FILTER>...<FILTER></FILTERS>

that is, a proper XML document.
With CQL the issue is the same... to properly handle nesting,
you need to throw the full spec to a single parser that
understands the documents as a whole.

Cheers
Andrea