[Geoserver-devel] WMS . SLD and filter

Hi,

a quick question - have been playing with a branch (complex-features) based on RC1.5 - and I've noticed that the application of Filters in SLD requests to WMS (using SLD_BODY) seems to apply the filter AFTER selecting all the data from the database, which is a little sad really when you want to display a few elements from millions...

I found the following "For WMS queries, the Filter parameters are handled in both PostGIS and
GeoServer. GeoServer makes a request to PostGIS for all the features
that intersect the view rectangle. It also makes a set of Rules that are
valid for the current scale range.
Once it has those features, it runs the filters on them on the GeoServer
side, as there might be several filters and else clauses."

from Brent which seems to imply its a design decision.

Is there any way around this currently? Any ideas on how hard it might be to fix this - say to apply the filter at the server side if there is only one Rule present? (The client can then at least send multiple SLD's with single rules - would fix the 80% cases and provide a workaround for the 20%)

Rob Atkinson

Hi,

For SQL datastores, I'd suggest that each Filter of a Rule in SLD should
(or could?) be translated into the where clause. For instance: select *
from foo where [spatialCondition] and ([conditionsOfRule1] or
[conditionsOfRule2] or [conditionsOfRule3]). This should return only the
data that is needed and only 1 SQL-statement is used.

But maybe it's easier to have one SQL-statement per Filter, because WFS is
working that way now. If one sends a
GetFeature-request for each featuretype an SQL-statement is constructed.
The results are combined in one featurecollection. Something similar could
be made for WMS/SLD I think.

In my opinion, rather not implement this for only one filter per SLD.
Would break with the SLD specs...

And what about the optimizedDataLoadingEnabled-option in
queryLayer(MapLayer currLayer, Envelope envelope) in LiteRenderer2? Is
that used or otherwise useful here?

Thijs

-----Original Message-----
From: Rob Atkinson <rob@anonymised.com>
To: Geoserver-devel <geoserver-devel@lists.sourceforge.net>
Date: Fri, 23 Dec 2005 23:59:36 +1100
Subject: [Geoserver-devel] WMS . SLD and filter

Hi,

a quick question - have been playing with a branch (complex-features)
based on RC1.5 - and I've noticed that the application of Filters in
SLD
requests to WMS (using SLD_BODY) seems to apply the filter AFTER
selecting all the data from the database, which is a little sad really
when you want to display a few elements from millions...

I found the following "For WMS queries, the Filter parameters are
handled in both PostGIS and
GeoServer. GeoServer makes a request to PostGIS for all the features
that intersect the view rectangle. It also makes a set of Rules that
are
valid for the current scale range.
Once it has those features, it runs the filters on them on the
GeoServer
side, as there might be several filters and else clauses."

from Brent which seems to imply its a design decision.

Is there any way around this currently? Any ideas on how hard it
might
be to fix this - say to apply the filter at the server side if there is
only one Rule present? (The client can then at least send multiple
SLD's
with single rules - would fix the 80% cases and provide a workaround
for
the 20%)

Rob Atkinson

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log
files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Quoting Thijs Brentjens <thijs.brentjens@anonymised.com>:

Hi,

For SQL datastores, I'd suggest that each Filter of a Rule in SLD
should
(or could?) be translated into the where clause. For instance: select
*
from foo where [spatialCondition] and ([conditionsOfRule1] or
[conditionsOfRule2] or [conditionsOfRule3]). This should return only
the
data that is needed and only 1 SQL-statement is used.

Yes, this improvement is needed. I think David said the reason we're
not doing it is because of the 'else' thing, which we don't have in
WFS, and which basically means you'd want everything - anything that
doesn't match any of the first filter rules. But I think we can be
smarter about it, as there are many times when people use SLD rules
that don't have an else, and some can get huge performance gains. I
think Alessio started to look into this, maybe he can let us know what
might need to be done.

But maybe it's easier to have one SQL-statement per Filter, because
WFS is
working that way now. If one sends a
GetFeature-request for each featuretype an SQL-statement is
constructed.
The results are combined in one featurecollection. Something similar
could
be made for WMS/SLD I think.

It's actually a lot better to make just one statement, as there's a
decent bit of overhead to make a db connection - the number of db
requests should be minimized as much as possible. I would actually say
that a WMS request as one filter mirrors a WFS getFeature request. If
you request multiple WMS layers, then we would have to make multiple
requests, but we shouldn't have to for a single layer. This is how WFS
works, for each featureType a sql statement is made, not for each
filter - you can have many filters anded and ored together. So too
should a single SLD based request figure out how to pass which filters
in, and figure out which filters to deal with in code. This will take
a decent bit of work, it's tricky code, but it should pay off in terms
of performance gains. I imagine it would take someone who knows the
code 3-5 days to make it solid. And I think you could maybe get some
quick performance gains, like for sld files that just have one rule,
relatively quickly.

In my opinion, rather not implement this for only one filter per SLD.
Would break with the SLD specs...

And what about the optimizedDataLoadingEnabled-option in
queryLayer(MapLayer currLayer, Envelope envelope) in LiteRenderer2?
Is
that used or otherwise useful here?

Looks like all that does is send the bounding box to the datastore, and
only get attributes that are used by the stylers. So it's already
being used in geoserver.

Chris

Thijs

-----Original Message-----
From: Rob Atkinson <rob@anonymised.com>
To: Geoserver-devel <geoserver-devel@lists.sourceforge.net>
Date: Fri, 23 Dec 2005 23:59:36 +1100
Subject: [Geoserver-devel] WMS . SLD and filter

> Hi,
>
> a quick question - have been playing with a branch
(complex-features)
> based on RC1.5 - and I've noticed that the application of Filters
in
> SLD
> requests to WMS (using SLD_BODY) seems to apply the filter AFTER
> selecting all the data from the database, which is a little sad
really
> when you want to display a few elements from millions...
>
>
>
> I found the following "For WMS queries, the Filter parameters are
> handled in both PostGIS and
> GeoServer. GeoServer makes a request to PostGIS for all the
features
> that intersect the view rectangle. It also makes a set of Rules
that
> are
> valid for the current scale range.
> Once it has those features, it runs the filters on them on the
> GeoServer
> side, as there might be several filters and else clauses."
>
> from Brent which seems to imply its a design decision.
>
> Is there any way around this currently? Any ideas on how hard it
> might
> be to fix this - say to apply the filter at the server side if
there is
> only one Rule present? (The client can then at least send multiple
> SLD's
> with single rules - would fix the 80% cases and provide a
workaround
> for
> the 20%)
>
> Rob Atkinson
>
>
>
>
>
>
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through
log
> files
> for problems? Stop! Download the new AJAX search engine that
makes
> searching your log files as easy as surfing the web. DOWNLOAD
SPLUNK!
> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
> _______________________________________________
> Geoserver-devel mailing list
> Geoserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geoserver-devel

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through
log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD
SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

----------------------------------------------------------
This mail sent through IMP: https://webmail.limegroup.com/

On 12/23/05, Chris Holmes <cholmes@...13...> wrote:

Quoting Thijs Brentjens <thijs.brentjens@...467...>:

> Hi,
>
> For SQL datastores, I'd suggest that each Filter of a Rule in SLD
> should
> (or could?) be translated into the where clause. For instance: select
> *
> from foo where [spatialCondition] and ([conditionsOfRule1] or
> [conditionsOfRule2] or [conditionsOfRule3]). This should return only
> the
> data that is needed and only 1 SQL-statement is used.
Yes, this improvement is needed. I think David said the reason we're
not doing it is because of the 'else' thing, which we don't have in
WFS, and which basically means you'd want everything - anything that
doesn't match any of the first filter rules. But I think we can be
smarter about it, as there are many times when people use SLD rules
that don't have an else, and some can get huge performance gains. I
think Alessio started to look into this, maybe he can let us know what
might need to be done.

As Chris implied, the translation of the Filter to SQL statement is
already present in WFS.

Currently, I use a hack (creating a view dynamically in the db) for
selecting only desired geometries during a WMS request. It is much
much quicker, esp for large tables. But what happens if 2 or more
people make a WMS request at the same time? Hmm...

I would love to be able to use a good working solution, where the
filter is part of the SQL statement.

Alex