Hello,
I use app-schema to create complex features for a dutch GML 3.2 schema. I cannot use it live because it is a bit slow, so I use a batch to create xml files.
I don’t know for sure, but it looks as if the app-schema opens a new database connection for every sub element. Since I have about 10, this means that every request opens 11 connections.
It looks like they do not get closed. Because after 50 requests to a feature, the database throws errors saying there are to many connections. It even hangs me server.
Can someone confirm that there are issues with database connections not being closed? Or should I post my question to dev? Is the question somewhat clear?
Kind regards,
Milo van der Linden
Milo,
apologies for the delayed reply: holiday season here in the antipodes.
What version of GeoServer are you using? Are you using joining?
I can confirm that app-schema will open a new database connection for every FeatureTypeMapping (possible less with joining). The reason for this is that app-schema re-uses the GeoServer simple feature implementation to access data that is then mapped into complex features. However, GeoServer will also pool and re-use connections. Any leaking of connections is a bug; please report. Please try increasing the number of connections permitted by your database.
Joining should reduce the number of connections used at any one time.
Use of a a JNDI connection pool allows you to configure all connection parameters and share connections between mappings (and even applications); this should give you the best use of connections in a production deployment:
http://docs.geoserver.org/latest/en/user/data/app-schema/data-stores.html#jndi
Kind regards,
Ben.
On 14/12/13 19:50, Milo van der Linden wrote:
Hello,
I use app-schema to create complex features for a dutch GML 3.2 schema.
I cannot use it live because it is a bit slow, so I use a batch to
create xml files.
I don't know for sure, but it looks as if the app-schema opens a new
database connection for every sub element. Since I have about 10, this
means that every request opens 11 connections.
It looks like they do not get closed. Because after 50 requests to a
feature, the database throws errors saying there are to many
connections. It even hangs me server.
Can someone confirm that there are issues with database connections not
being closed? Or should I post my question to dev? Is the question
somewhat clear?
Kind regards,
Milo van der Linden
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre
On Mon, Jan 13, 2014 at 5:00 AM, Ben Caradoc-Davies <
Ben.Caradoc-Davies@anonymised.com> wrote:
Milo,
apologies for the delayed reply: holiday season here in the antipodes.
What version of GeoServer are you using? Are you using joining?
I can confirm that app-schema will open a new database connection for
every FeatureTypeMapping (possible less with joining). The reason for
this is that app-schema re-uses the GeoServer simple feature
implementation to access data that is then mapped into complex features.
However, GeoServer will also pool and re-use connections. Any leaking of
connections is a bug; please report. Please try increasing the number of
connections permitted by your database.
An observation here. Say all, or most of your simple feature types are
coming
from the same database. And say also, that app-schema does not use any
form of multithreading when fetching the data.
In that case if you make all your reads in the same GeoTools transaction,
then they will all use the same connection, having multiple result sets
open at the same time is legal, provided you read from the using a single
thread.
Cheers
Andrea
--
== Our support, Your Success! Visit http://opensdi.geo-solutions.it for
more information ==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
Thank you Andrea and Ben for responding!
I will run some tests later this week. Theoretically I follow what both of you describe, but I am getting postgresql errors that there are too many database connections. I want to dig up some solid proof and will try the proposed tweaks.
My geoserver version is 2.5 and the database is postgresql 8.3 with postgis 1.5 although I am planning to migrate to postgresql 9.3 and postgis 2.1 on short notice.
Thanks again!
Kind regards,
Milo
···
2014/1/13 Andrea Aime <andrea.aime@anonymised.com>
On Mon, Jan 13, 2014 at 5:00 AM, Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com> wrote:
Milo,
apologies for the delayed reply: holiday season here in the antipodes.
What version of GeoServer are you using? Are you using joining?
I can confirm that app-schema will open a new database connection for
every FeatureTypeMapping (possible less with joining). The reason for
this is that app-schema re-uses the GeoServer simple feature
implementation to access data that is then mapped into complex features.
However, GeoServer will also pool and re-use connections. Any leaking of
connections is a bug; please report. Please try increasing the number of
connections permitted by your database.
An observation here. Say all, or most of your simple feature types are coming
from the same database. And say also, that app-schema does not use any
form of multithreading when fetching the data.
In that case if you make all your reads in the same GeoTools transaction,
then they will all use the same connection, having multiple result sets
open at the same time is legal, provided you read from the using a single
thread.
Cheers
Andrea
–
== Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information ==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
Thanks very much, Andrea. That certainly sounds like an interesting improvement. Each app-schema request runs in a single thread so this should be possible (Rini knows feature chaining best and has the final word). One exception is the WFS 2.0.0 resolve timeout which uses a separate thread; this might require special consideration.
Kind regards,
Ben.
On 13/01/14 16:17, Andrea Aime wrote:
An observation here. Say all, or most of your simple feature types are
coming
from the same database. And say also, that app-schema does not use any
form of multithreading when fetching the data.
In that case if you make all your reads in the same GeoTools transaction,
then they will all use the same connection, having multiple result sets
open at the same time is legal, provided you read from the using a single
thread.
--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre
On Mon, Jan 13, 2014 at 9:48 AM, Ben Caradoc-Davies <
Ben.Caradoc-Davies@anonymised.com> wrote:
Thanks very much, Andrea. That certainly sounds like an interesting
improvement. Each app-schema request runs in a single thread so this should
be possible (Rini knows feature chaining best and has the final word). One
exception is the WFS 2.0.0 resolve timeout which uses a separate thread;
this might require special consideration.
Mind, to set the transaction you'll have to cast them to FeatureStore...
which they normally implement anyways, unless they don't have a primary key
(but in that case I guess you might have a hard time handling them?)
But we should consider moving getTransaction/setTransaction to
FeatureSource, as reads from a feature type can be interspersed with writes
on other feature types in the same store.
Or, you make app-schema call DataStore.getReader(query, transaction) to
have reads involved in a transaction
Cheers
Andrea
--
== Our support, Your Success! Visit http://opensdi.geo-solutions.it for
more information ==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------