[Geoserver-users] App schema efficiency

Dear all,

I am trying to implement an app-schema for a complex data type. In my test example are 2 tables, a container table and a content table in a one to one relationship (in the complete example, there are one to many and many to many relationships).

What I am trying to accomplish is an efficient search for the content table. I run a query something like this:

<wfs:GetFeature service=“WFS” version=“1.1.0” xmlns:gsml=“urn:cgi:xmlns:CGI:GeoSciML:2.0”
xmlns:wfs=“http://www.opengis.net/wfs” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance
xsi:schemaLocation=“http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.1.0/wfs.xsd
xmlns:ogc=“http://www.opengis.net/ogc”>
<wfs:Query typeName=“test:container”>
ogc:Filter
ogc:PropertyIsEqualTo
ogc:PropertyNametest:containerlinkelement/test:contenttable/test:contentlabel</ogc:PropertyName>
ogc:Literal8889</ogc:Literal>
</ogc:PropertyIsEqualTo>
</ogc:Filter>
</wfs:Query>

But what I see is that he runs 2 SQL queries:

  1. A query for the entire container table
  2. A query with a join with the content table

It then proceeds to iterate over all the elements causing significant delay. I was expecting a join with a where clause. Is this normal behaviour, or have I done something wrong.

Thank you in advance,

Lennert

Lennart,

what version of GeoServer are you using?

app-schema joining (efficient SQL queries) is is turned on by default in recent GeoServer releases;
http://docs.geoserver.org/latest/en/user/data/app-schema/joining.html

Joining can be prevented if you are using feature chaining across different database or if you are making filter queries based on properties that are created in GeoServer using CQL expressions in your mapping file (for example, string concatenation).

Kind regards,
Ben.

On 07/03/14 17:44, Lennert De Feyter wrote:

Dear all,

I am trying to implement an app-schema for a complex data type. In my
test example are 2 tables, a container table and a content table in a
one to one relationship (in the complete example, there are one to many
and many to many relationships).

What I am trying to accomplish is an efficient search for the content
table. I run a query something like this:

<wfs:GetFeature service="WFS" version="1.1.0"
xmlns:gsml="urn:cgi:xmlns:CGI:GeoSciML:2.0"
     xmlns:wfs="http://www.opengis.net/wfs&quot;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot;

xsi:schemaLocation="http://www.opengis.net/wfshttp://schemas.opengis.net/wfs/1.1.0/wfs.xsd&quot;

     xmlns:ogc="http://www.opengis.net/ogc&quot;&gt;
     <wfs:Query typeName="test:container">
        <ogc:Filter>
<ogc:PropertyIsEqualTo>

<ogc:PropertyName>test:containerlinkelement/test:contenttable/test:contentlabel</ogc:PropertyName>

<ogc:Literal>8889</ogc:Literal>
          </ogc:PropertyIsEqualTo>
       </ogc:Filter>
       </wfs:Query>

But what I see is that he runs 2 SQL queries:
1) A query for the entire container table
2) A query with a join with the content table

It then proceeds to iterate over all the elements causing significant
delay. I was expecting a join with a where clause. Is this normal
behaviour, or have I done something wrong.

Thank you in advance,

Lennert

------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries. Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk

_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

It sounds like joining is turned on, i.e. only 2 queries run, but the filter involves nested attributes from another table.
This kind of filter currently isn't incorporated into the SQL query yet, thus the inefficient iterating through the features.
This is a gap we're looking to fix. The current workaround sadly is to limit the number of matches (maxFeatures for WFS 1.1.0, count for WFS 2.0).

Regards
Rini

-----Original Message-----
From: Caradoc-Davies, Ben (CESRE, Kensington)
Sent: Monday, 10 March 2014 9:31 AM
To: Lennert De Feyter; geoserver-users@lists.sourceforge.net
Cc: Angreani, Rini (CESRE, Kensington)
Subject: Re: [Geoserver-users] App schema efficiency

Lennart,

what version of GeoServer are you using?

app-schema joining (efficient SQL queries) is is turned on by default in recent GeoServer releases; http://docs.geoserver.org/latest/en/user/data/app-schema/joining.html

Joining can be prevented if you are using feature chaining across different database or if you are making filter queries based on properties that are created in GeoServer using CQL expressions in your mapping file (for example, string concatenation).

Kind regards,
Ben.

On 07/03/14 17:44, Lennert De Feyter wrote:

Dear all,

I am trying to implement an app-schema for a complex data type. In my
test example are 2 tables, a container table and a content table in a
one to one relationship (in the complete example, there are one to
many and many to many relationships).

What I am trying to accomplish is an efficient search for the content
table. I run a query something like this:

<wfs:GetFeature service="WFS" version="1.1.0"
xmlns:gsml="urn:cgi:xmlns:CGI:GeoSciML:2.0"
     xmlns:wfs="http://www.opengis.net/wfs&quot;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot;

xsi:schemaLocation="http://www.opengis.net/wfshttp://schemas.opengis.net/wfs/1.1.0/wfs.xsd&quot;

     xmlns:ogc="http://www.opengis.net/ogc&quot;&gt;
     <wfs:Query typeName="test:container">
        <ogc:Filter>
<ogc:PropertyIsEqualTo>

<ogc:PropertyName>test:containerlinkelement/test:contenttable/test:con
tentlabel</ogc:PropertyName>

<ogc:Literal>8889</ogc:Literal>
          </ogc:PropertyIsEqualTo>
       </ogc:Filter>
       </wfs:Query>

But what I see is that he runs 2 SQL queries:
1) A query for the entire container table
2) A query with a join with the content table

It then proceeds to iterate over all the elements causing significant
delay. I was expecting a join with a where clause. Is this normal
behaviour, or have I done something wrong.

Thank you in advance,

Lennert

----------------------------------------------------------------------
-------- Subversion Kills Productivity. Get off Subversion & Make the
Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries. Built-in WAN optimization
and the freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.
clktrk

_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@...367...> Software Engineer CSIRO Earth Science and Resource Engineering Australian Resources Research Centre

Alternatively, you could run your filter with different bounding box (with tiling), so the filtering is done on small subset each time.

-----Original Message-----
From: Angreani, Rini (CESRE, Kensington)
Sent: Monday, 10 March 2014 9:58 AM
To: Caradoc-Davies, Ben (CESRE, Kensington); Lennert De Feyter; geoserver-users@lists.sourceforge.net
Subject: RE: [Geoserver-users] App schema efficiency

It sounds like joining is turned on, i.e. only 2 queries run, but the filter involves nested attributes from another table.
This kind of filter currently isn't incorporated into the SQL query yet, thus the inefficient iterating through the features.
This is a gap we're looking to fix. The current workaround sadly is to limit the number of matches (maxFeatures for WFS 1.1.0, count for WFS 2.0).

Regards
Rini

-----Original Message-----
From: Caradoc-Davies, Ben (CESRE, Kensington)
Sent: Monday, 10 March 2014 9:31 AM
To: Lennert De Feyter; geoserver-users@lists.sourceforge.net
Cc: Angreani, Rini (CESRE, Kensington)
Subject: Re: [Geoserver-users] App schema efficiency

Lennart,

what version of GeoServer are you using?

app-schema joining (efficient SQL queries) is is turned on by default in recent GeoServer releases; http://docs.geoserver.org/latest/en/user/data/app-schema/joining.html

Joining can be prevented if you are using feature chaining across different database or if you are making filter queries based on properties that are created in GeoServer using CQL expressions in your mapping file (for example, string concatenation).

Kind regards,
Ben.

On 07/03/14 17:44, Lennert De Feyter wrote:

Dear all,

I am trying to implement an app-schema for a complex data type. In my
test example are 2 tables, a container table and a content table in a
one to one relationship (in the complete example, there are one to
many and many to many relationships).

What I am trying to accomplish is an efficient search for the content
table. I run a query something like this:

<wfs:GetFeature service="WFS" version="1.1.0"
xmlns:gsml="urn:cgi:xmlns:CGI:GeoSciML:2.0"
     xmlns:wfs="http://www.opengis.net/wfs&quot;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot;

xsi:schemaLocation="http://www.opengis.net/wfshttp://schemas.opengis.net/wfs/1.1.0/wfs.xsd&quot;

     xmlns:ogc="http://www.opengis.net/ogc&quot;&gt;
     <wfs:Query typeName="test:container">
        <ogc:Filter>
<ogc:PropertyIsEqualTo>

<ogc:PropertyName>test:containerlinkelement/test:contenttable/test:con
tentlabel</ogc:PropertyName>

<ogc:Literal>8889</ogc:Literal>
          </ogc:PropertyIsEqualTo>
       </ogc:Filter>
       </wfs:Query>

But what I see is that he runs 2 SQL queries:
1) A query for the entire container table
2) A query with a join with the content table

It then proceeds to iterate over all the elements causing significant
delay. I was expecting a join with a where clause. Is this normal
behaviour, or have I done something wrong.

Thank you in advance,

Lennert

----------------------------------------------------------------------
-------- Subversion Kills Productivity. Get off Subversion & Make the
Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries. Built-in WAN optimization
and the freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.
clktrk

_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@...367...> Software Engineer CSIRO Earth Science and Resource Engineering Australian Resources Research Centre

Joining is indeed turned on.

Thank you all for the replies. At least I now know I didn’t do anything wrong.

Lennert