We (Dept. Primary Industry, Vic, Aus > Earth Resources Division) have deployed GeoServer 2.2 with app-schema module on Tomcat 6 serving GeoSciML 2 & 3 into UAT and Production Servers.
Our main obstacle to taking these services beyond just experimenting & trialling is performance issues.
Currently no server instance has done better than a 4 second response time for a single feature GetFeature request where the ID of the Feature was specified in the request.
And for more general GetFeature requests (such as using maxFeatures or bounding boxes) the performance is considerably worse.
I have been analyzing the GeoServer instances in a profiler and run several debug sessions while profiling.
One particular concern arose when I followed the execution in the FilteringFeatureReader class.
To try and put this into context, this class was called/used when doing a request for a GeologicalUnit feature which had a GeologicalEvent linked to it using a foreign key relationship. It’s a many to one relationship - many GeologicalEvents can be associated with a GeologicalUnit feature and hence the foreign key is in the GeologicalEvent table.
(I can put the relevant app-schema configuration that has the FEATURE_LINK relationship in later.)
NOW THE PERFORMANCE ISSUE IS, that GeoTools does an SQL fetch without a where clause on the GeologicalEvent table, thus fetching the entire table,
and then iterates through the entire result set (using the FilteringFeatureReader class) comparing whether the row is the related GeologicalEvent for the requested GeologicalUnit.
The GeologicalEvent table has tens of thousands of records in it and this part of the request processing is a major bottleneck contributing (I think) more than anything else to the blow-out in the response times.
I was expecting that if all the foreign key relationships were specified in the app-schema configuration, then I would see only SQL statements in the profiler that have WHERE clauses for matching related rows in our Oracle database.
There is no attribute filtering/searching in the request but even if there were, I would have thought the associated features (we’re dealing here with complex features and feature chaining) could still be extracted using appropriate where clauses.
No doubt there is a reason for the FilteringFeatureReader class doing what it’s doing.
Can you help with finding a way to eliminate it being called in the request/response processing code.
Is there any general advice on this.
Could having a more optimal app-schema configuration help.
I can post the app-schema configuration files or relevant portions of them if you think it could illuminate a problem.
For backwards compatibility, app-schema joining support is disabled by default. Enabling joining for performance is strongly recommended, and is particularly important for feature types with multivalued properties. With joining enabled, more efficient SQL queries are generated and performance typically improves for large data sets, sometimes by several orders of magnitude.
Kind regards,
Ben.
On 06/02/13 09:09, Gavin wrote:
We (Dept. Primary Industry, Vic, Aus > Earth Resources Division)
have deployed GeoServer 2.2 with app-schema module on Tomcat 6 serving
GeoSciML 2 & 3 into UAT and Production Servers.
Our main obstacle to taking these services beyond just experimenting &
trialling is performance issues.
Currently no server instance has done better than a 4 second response
time for a single feature GetFeature request where the ID of the Feature
was specified in the request.
And for more general GetFeature requests (such as using maxFeatures or
bounding boxes) the performance is considerably worse.
I have been analyzing the GeoServer instances in a profiler and run
several debug sessions while profiling.
One particular concern arose when I followed the execution in the
FilteringFeatureReader class.
To try and put this into context, this class was called/used when doing
a request for a GeologicalUnit feature which had a GeologicalEvent
linked to it using a foreign key relationship. It's a many to one
relationship - many GeologicalEvents can be associated with a
GeologicalUnit feature and hence the foreign key is in the
GeologicalEvent table.
(I can put the relevant app-schema configuration that has the
FEATURE_LINK relationship in later.)
NOW THE PERFORMANCE ISSUE IS, that GeoTools does an SQL fetch without a
where clause on the GeologicalEvent table, thus fetching the entire table,
and then iterates through the entire result set (using the
FilteringFeatureReader class) comparing whether the row is the related
GeologicalEvent for the requested GeologicalUnit.
The GeologicalEvent table has tens of thousands of records in it and
this part of the request processing is a major bottleneck contributing
(I think) more than anything else to the blow-out in the response times.
I was expecting that if all the foreign key relationships were specified
in the app-schema configuration, then I would see only SQL statements in
the profiler that have WHERE clauses for matching related rows in our
Oracle database.
There is no attribute filtering/searching in the request but even if
there were, I would have thought the associated features (we're dealing
here with complex features and feature chaining) could still be
extracted using appropriate where clauses.
No doubt there is a reason for the FilteringFeatureReader class doing
what it's doing.
Can you help with finding a way to eliminate it being called in the
request/response processing code.
Is there any general advice on this.
Could having a more optimal app-schema configuration help.
I can post the app-schema configuration files or relevant portions of
them if you think it could illuminate a problem.
--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre
For backwards compatibility, app-schema joining support is disabled by
default.
Ben,
wouldn’t it be better to enable it by default in some future version, warning
people that the defaults changed, and that they can turn it off if they wish to?
On Wed, Feb 6, 2013 at 4:07 AM, Ben Caradoc-Davies
<Ben.Caradoc-Davies@anonymised.com <mailto:Ben.Caradoc-Davies@anonymised.com>> wrote:
For backwards compatibility, app-schema joining support is disabled by
default.
wouldn't it be better to enable it by default in some future version,
warning
people that the defaults changed, and that they can turn it off if they
wish to?
Joining has now been in use long enough that this should be given serious consideration.
Rini, do you think joining should be enabled by default?
Kind regards,
--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre
I actually thought about it. We should do it now that more people (should) use joining.
-----Original Message-----
From: Caradoc-Davies, Ben (CESRE, Kensington)
Sent: Wednesday, 6 February 2013 3:56 PM
To: Andrea Aime; Angreani, Rini (CESRE, Kensington)
Cc: Gavin; Geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] WFS Performance Issues with App-schema Module
On 06/02/13 15:09, Andrea Aime wrote:
On Wed, Feb 6, 2013 at 4:07 AM, Ben Caradoc-Davies
<Ben.Caradoc-Davies@...367... <mailto:Ben.Caradoc-Davies@…367…>> wrote:
For backwards compatibility, app-schema joining support is disabled by
default.
wouldn't it be better to enable it by default in some future version,
warning people that the defaults changed, and that they can turn it
off if they wish to?
Joining has now been in use long enough that this should be given serious consideration.
Rini, do you think joining should be enabled by default?
Kind regards,
--
Ben Caradoc-Davies <Ben.Caradoc-Davies@...367...> Software Engineer CSIRO Earth Science and Resource Engineering Australian Resources Research Centre