Andrea,
First, I forgot to say this in on GeoServer 2.6.2, postgres 9.3.6, Linux Ubuntu 14.04. JNDI is NOT in use at this point. Just 2 mosaics on this machine. Nothing else defined or in use.
The error IS connection related … which is what I think you were asking related to leaking connections from the pool. I am seeing an error saying that a connection can’t be established … “FATAL: remaining connection slots are reserved for non-replication superuser connections” … so the connections are exhausted. Partial stack trace is below, and the very base of the error is in getConnection() … so the connection can’t be made.
ERROR [gce.imagemosaic] -
java.io.IOException
at org.geotools.gce.imagemosaic.CatalogManager.createGranuleCatalogFromDatastore(CatalogManage
r.java:200)
… [omitting full trace - can send if you need it] …
… 103 more
Caused by: org.postgresql.util.PSQLException: FATAL: remaining connection slots are reserved for non-replication superuser connections
at org.postgresql.core.v3.ConnectionFactoryImpl.readStartupMessages(ConnectionFactoryImpl.java:464)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:112)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
at org.postgresql.jdbc2.AbstractJdbc2Connection.(AbstractJdbc2Connection.java:125)
at org.postgresql.jdbc3.AbstractJdbc3Connection.(AbstractJdbc3Connection.java:30)
at org.postgresql.jdbc3.Jdbc3Connection.(Jdbc3Connection.java:24)
at org.postgresql.Driver.makeConnection(Driver.java:393)
at org.postgresql.Driver.connect(Driver.java:267)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at org.geotools.data.postgis.PostgisNGDataStoreFactory.getConnection(PostgisNGDataStoreFactory.java:272)
at org.geotools.data.postgis.PostgisNGDataStoreFactory.createDataSource(PostgisNGDataStoreFactory.java:222)
… 106 more
As far as the mosaics … as I mentioned, this is just for 2 image mosaics. There is nothing else on this box at this point (except one vector store with one layer that’s not being used). I thought the default max connections in each pool was for 10 per store, but have experimented with max\ connections set to 5, and still very quickly reach the 98 connection limit on postgres … ALL from just harvesting mosaics sequentially (not entire directory at once) in these 2 stores. So, the 98 connections I showed are all created from harvesting in these 2 stores.
My current datastore.properties for both stores is:
SPI=org.geotools.data.postgis.PostgisNGDataStoreFactory
host=localhost
port=5432
database=******
schema=mosaics
user=******
passwd=******
max\ connections=5
Loose\ bbox=true
Estimated\ extends=false
validate\ connections=true
Connection\ timeout=10
preparedStatements=true
So, does this not seem like connections are being leaked somewhere? Mosaic harvesting not using the pool?
I wanted to break out of my script and make sure there wasn’t anything wrong with my workflow. So, I created the simplest mosaic I could with the same datastore.properties file posted above. It consisted of 2 temperature files from the geosolutions multidimensional tutorial in a directory called "casefour’.
When I run the following command at the command line:
curl -v -u xxxxx:xxxxx -XPOST -H “Content-type: text/plain” -d “file:/home/ubuntu/rest_harvesting/casefour” http://localhost:8080/geoserver/rest/workspaces/geosolutions/coveragestores/casefour/external.imagemosaic
there is a 1 : 1 relationship every time I execute this command to a new connection being opened to the database and remaining idle afterwards. If I run that command 90 times, I end up with 90 connections open in the db. max\ connections is set to 5 (default 10, right) … so why should I see more than 5 connections?
What else can I do to troubleshoot and help figure this out?
Thanks,
Mike
···
On Sat, Apr 11, 2015 at 2:20 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:
On Fri, Apr 10, 2015 at 11:37 PM, Mike Grogan <d.michael.grogan@anonymised.com…> wrote:
I see “select now()” being used as a validationQuery in lots of places in the source. It appears, though, that database connections are not being closed somewhere after harvesting via REST calls.
GeoServer uses connection pools, so it’s normal that they are not closed, they are supposedly
just given back to the pool.
We need to understand if we are leaking them (not releasing them back to the pool), or if it’s just
that you setup the pool with too many connections, compared to what your dbms is willing
to support.
If it’s the first, you’ll see the code fail with an error stating it was not possible to get a connection
from the pool… that does not seem your case.
If it’s the latter, we probably need to understand better your setup, how many mosaics, your
datastore.properties, and so on.
In general, if you have many dbms based mosaics, you will have a separate pool for each, which will
quickly exhaust your connections server side, unless you setup a shared connection pool at the
tomcat level, expose it via jndi, and refer to it that way.
Examples here:
http://geoserver.geo-solutions.it/multidim/en/imagemosaic/mosaic_datastore.html#using-a-jndi-connection-pool
Cheers
Andrea
–
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.