[Geoserver-users] postgres connection exhaustion - Image Mosaic Harvesting

I am experiencing an exhaustion of postgres connections after/during harvesting multiple files (sequentially, not simultaneously) with calls via the GeoServer REST API. The culprit seems to be a lot of select now() queries, on connections which seem to never close and which quickly lead to “org.postgresql.util.PSQLException: FATAL: remaining connection slots are reserved for non-replication superuser connections” errors.

My pg_stat_activity looks like this before harvesting files (2 total rows):

postgres=# select datid, datname, pid, client_addr, client_port, state, query from pg_stat_activity;
datid | datname | pid | client_addr | client_port | state | query
-------±---------±------±------------±------------±-------±------------------------------------------------------------------------------------------
19062 | basegis | 24675 | 127.0.0.1 | 39598 | idle | select now()
12035 | postgres | 24729 | | -1 | active | select datid, datname, pid, client_addr, client_port, state, query from pg_stat_activity;
(2 rows)

It looks like this after harvesting files (98 total rows – connections exhausted):

datid | datname | pid | client_addr | client_port | state | query
-------±-----------±------±------------±------------±-------±------------------------------------------------------------------------------------------
19062 | basegis | 24675 | 127.0.0.1 | 39598 | idle | select now()
17688 | stormprint | 25669 | 127.0.0.1 | 40072 | idle | select now()
17688 | stormprint | 24869 | 127.0.0.1 | 39602 | idle | COMMIT
17688 | stormprint | 25671 | 127.0.0.1 | 40074 | idle | select now()
17688 | stormprint | 24872 | 127.0.0.1 | 39605 | idle | COMMIT
17688 | stormprint | 25675 | 127.0.0.1 | 40076 | idle | select now()
17688 | stormprint | 25677 | 127.0.0.1 | 40078 | idle | select now()
17688 | stormprint | 25681 | 127.0.0.1 | 40080 | idle | select now()
17688 | stormprint | 25683 | 127.0.0.1 | 40082 | idle | select now()
17688 | stormprint | 25687 | 127.0.0.1 | 40084 | idle | select now()

… [multiple similar rows omitted] …

17688 | stormprint | 25930 | 127.0.0.1 | 40251 | idle | select now()

17688 | stormprint | 25934 | 127.0.0.1 | 40253 | idle | select now()
17688 | stormprint | 25936 | 127.0.0.1 | 40255 | idle | select now()
17688 | stormprint | 25940 | 127.0.0.1 | 40257 | idle | select now()
12035 | postgres | 26035 | | -1 | active | select datid, datname, pid, client_addr, client_port, state, query from pg_stat_activity;
(98 rows)

I see “select now()” being used as a validationQuery in lots of places in the source. It appears, though, that database connections are not being closed somewhere after harvesting via REST calls.

Thanks,

Mike Grogan

On Fri, Apr 10, 2015 at 11:37 PM, Mike Grogan <d.michael.grogan@anonymised.com>
wrote:

I see "select now()" being used as a validationQuery in lots of places in
the source. It appears, though, that database connections are not being
closed somewhere after harvesting via REST calls.

GeoServer uses connection pools, so it's normal that they are not closed,
they are supposedly
just given back to the pool.

We need to understand if we are leaking them (not releasing them back to
the pool), or if it's just
that you setup the pool with too many connections, compared to what your
dbms is willing
to support.

If it's the first, you'll see the code fail with an error stating it was
not possible to get a connection
from the pool... that does not seem your case.

If it's the latter, we probably need to understand better your setup, how
many mosaics, your
datastore.properties, and so on.
In general, if you have many dbms based mosaics, you will have a separate
pool for each, which will
quickly exhaust your connections server side, unless you setup a shared
connection pool at the
tomcat level, expose it via jndi, and refer to it that way.
Examples here:
http://geoserver.geo-solutions.it/multidim/en/imagemosaic/mosaic_datastore.html#using-a-jndi-connection-pool

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

Andrea,

First, I forgot to say this in on GeoServer 2.6.2, postgres 9.3.6, Linux Ubuntu 14.04. JNDI is NOT in use at this point. Just 2 mosaics on this machine. Nothing else defined or in use.

The error IS connection related … which is what I think you were asking related to leaking connections from the pool. I am seeing an error saying that a connection can’t be established … “FATAL: remaining connection slots are reserved for non-replication superuser connections” … so the connections are exhausted. Partial stack trace is below, and the very base of the error is in getConnection() … so the connection can’t be made.

ERROR [gce.imagemosaic] -

java.io.IOException
at org.geotools.gce.imagemosaic.CatalogManager.createGranuleCatalogFromDatastore(CatalogManage
r.java:200)

… [omitting full trace - can send if you need it] …

… 103 more
Caused by: org.postgresql.util.PSQLException: FATAL: remaining connection slots are reserved for non-replication superuser connections
at org.postgresql.core.v3.ConnectionFactoryImpl.readStartupMessages(ConnectionFactoryImpl.java:464)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:112)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
at org.postgresql.jdbc2.AbstractJdbc2Connection.(AbstractJdbc2Connection.java:125)
at org.postgresql.jdbc3.AbstractJdbc3Connection.(AbstractJdbc3Connection.java:30)
at org.postgresql.jdbc3.Jdbc3Connection.(Jdbc3Connection.java:24)
at org.postgresql.Driver.makeConnection(Driver.java:393)
at org.postgresql.Driver.connect(Driver.java:267)
at java.sql.DriverManager.getConnection(DriverManager.java:571)

at org.geotools.data.postgis.PostgisNGDataStoreFactory.getConnection(PostgisNGDataStoreFactory.java:272)
at org.geotools.data.postgis.PostgisNGDataStoreFactory.createDataSource(PostgisNGDataStoreFactory.java:222)
… 106 more

As far as the mosaics … as I mentioned, this is just for 2 image mosaics. There is nothing else on this box at this point (except one vector store with one layer that’s not being used). I thought the default max connections in each pool was for 10 per store, but have experimented with max\ connections set to 5, and still very quickly reach the 98 connection limit on postgres … ALL from just harvesting mosaics sequentially (not entire directory at once) in these 2 stores. So, the 98 connections I showed are all created from harvesting in these 2 stores.

My current datastore.properties for both stores is:

SPI=org.geotools.data.postgis.PostgisNGDataStoreFactory
host=localhost
port=5432
database=******
schema=mosaics
user=******
passwd=******
max\ connections=5
Loose\ bbox=true
Estimated\ extends=false
validate\ connections=true
Connection\ timeout=10
preparedStatements=true

So, does this not seem like connections are being leaked somewhere? Mosaic harvesting not using the pool?

I wanted to break out of my script and make sure there wasn’t anything wrong with my workflow. So, I created the simplest mosaic I could with the same datastore.properties file posted above. It consisted of 2 temperature files from the geosolutions multidimensional tutorial in a directory called "casefour’.

When I run the following command at the command line:

curl -v -u xxxxx:xxxxx -XPOST -H “Content-type: text/plain” -d “file:/home/ubuntu/rest_harvesting/casefour” http://localhost:8080/geoserver/rest/workspaces/geosolutions/coveragestores/casefour/external.imagemosaic

there is a 1 : 1 relationship every time I execute this command to a new connection being opened to the database and remaining idle afterwards. If I run that command 90 times, I end up with 90 connections open in the db. max\ connections is set to 5 (default 10, right) … so why should I see more than 5 connections?

What else can I do to troubleshoot and help figure this out?

Thanks,

Mike

···

On Sat, Apr 11, 2015 at 2:20 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Fri, Apr 10, 2015 at 11:37 PM, Mike Grogan <d.michael.grogan@anonymised.com…> wrote:

I see “select now()” being used as a validationQuery in lots of places in the source. It appears, though, that database connections are not being closed somewhere after harvesting via REST calls.

GeoServer uses connection pools, so it’s normal that they are not closed, they are supposedly
just given back to the pool.

We need to understand if we are leaking them (not releasing them back to the pool), or if it’s just
that you setup the pool with too many connections, compared to what your dbms is willing
to support.

If it’s the first, you’ll see the code fail with an error stating it was not possible to get a connection
from the pool… that does not seem your case.

If it’s the latter, we probably need to understand better your setup, how many mosaics, your
datastore.properties, and so on.
In general, if you have many dbms based mosaics, you will have a separate pool for each, which will
quickly exhaust your connections server side, unless you setup a shared connection pool at the
tomcat level, expose it via jndi, and refer to it that way.
Examples here:
http://geoserver.geo-solutions.it/multidim/en/imagemosaic/mosaic_datastore.html#using-a-jndi-connection-pool

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Sat, Apr 11, 2015 at 10:17 PM, Mike Grogan <d.michael.grogan@anonymised.com>
wrote:

As far as the mosaics ... as I mentioned, this is just for 2 image
mosaics. There is nothing else on this box at this point (except one
vector store with one layer that's not being used). I thought the default
max connections in each pool was for 10 per store, but have experimented
with max\ connections set to 5, and still very quickly reach the 98
connection limit on postgres ... ALL from just harvesting mosaics
sequentially (not entire directory at once) in these 2 stores. So, the 98
connections I showed are all created from harvesting in these 2 stores.

We are running production setups with more complex situations (more
mosaics, more traffic), so I guess we need to figure out what's making
yours different... I guess it's the lack of JNDI usage, we only do
production setups with JNDI because normally we have many mosaics
(and if we don't, each mosaic would have its own private pool).

From what I see, assuming GeoServer is the only application that can chew

away connections, we are probably leaking pools... are you calling
reloads in your scripts?

My current datastore.properties for both stores is:

SPI=org.geotools.data.postgis.PostgisNGDataStoreFactory
host=localhost
port=5432
database=******
schema=mosaics
user=******
passwd=******
max\ connections=5
Loose\ bbox=true
Estimated\ extends=false
validate\ connections=true
Connection\ timeout=10
preparedStatements=true

So, does this not seem like connections are being leaked somewhere?
Mosaic harvesting not using the pool?

As far as I know, that's not possible, there is no way to dodge it.

I wanted to break out of my script and make sure there wasn't anything
wrong with my workflow. So, I created the simplest mosaic I could with the
same datastore.properties file posted above. It consisted of 2 temperature
files from the geosolutions multidimensional tutorial in a directory called
"casefour'.

When I run the following command at the command line:

curl -v -u xxxxx:xxxxx -XPOST -H "Content-type: text/plain" -d
"file:/home/ubuntu/rest_harvesting/casefour"
http://localhost:8080/geoserver/rest/workspaces/geosolutions/coveragestores/casefour/external.imagemosaic

there is a 1 : 1 relationship every time I execute this command to a new
connection being opened to the database and remaining idle afterwards. If
I run that command 90 times, I end up with 90 connections open in the db.
max\ connections is set to 5 (default 10, right) ... so why should I see
more than 5 connections?

Indeed I don't know (off the top of my head at least). Daniele will
hopefully chime in an add some extra insight

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

On Sun, Apr 12, 2015 at 3:16 AM, Andrea Aime <andrea.aime@anonymised.com>
wrote:

We are running production setups with more complex situations (more
mosaics, more traffic), so I guess we need to figure out what's making
yours different... I guess it's the lack of JNDI usage, we only do
production setups with JNDI because normally we have many mosaics
(and if we don't, each mosaic would have its own private pool).

My plan was to switch to JNDI, but was just doing non-JNDI in development
and experienced this. I have switched to JNDI and am experiencing a
different issue, and am e-mailing about that in a different thread. Even
if I switch to JNDI, this issue would remain for those who don't.

From what I see, assuming GeoServer is the only application that can chew
away connections, we are probably leaking pools... are you calling
reloads in your scripts?

Yes, GeoServer is the only application running that can chew away at
connections. As far as reloads in the script, do you mean catalog
reloads? Just whatever gsconfig does when you save a resource. HOWEVER,
please see the part of my e-mail where I tested this with curl from the
command line completely outside of my script. There was certainly no
reload there. Just one REST command issued ... and a new database
connection established by GeoServer every time I ran that command.

Thanks,

Mike