[Geoserver-users] ArcSDE Connection Pooling

Brock_Anderson · July 10, 2007, 6:19pm

Hi All,

I'm seeing the following error in my Geoserver log file. Looks like Geoserver tries to go beyond the maximum number of ArcSDE connections it's configured for. Any thoughts on why that might be?

Some details about my setup:
-Geoserver 1.3.2
-Tomcat
-Windows 2003
-ArcSDE 9.1

I know there are newer versions of Geoserver now, but it's a significant effort for us to merely upgrade. I wonder if this is a known problem, and whether there might be an easier fix. Is an upgrade likely to help?

Brock

Caused by: org.geotools.data.DataSourceException: Problem with feature reader: Can't obtain a connection: The maximun of 8 to org.geotools.data.arcsde.ConnectionConfig[dbtype=arcsde, server=*****, port=*****, instance=*****, user=*****, password=*****, minConnections=2, maxConnections=8, connTimeOut=10000, connIncrement=1] has been reached
at org.geotools.data.arcsde.ArcSDEDataStore.getFeatureReader(ArcSDEDataStore.java:598)
at org.geotools.data.arcsde.ArcSDEDataStore.getFeatureReader(ArcSDEDataStore.java:612)
at org.geotools.data.DefaultFeatureResults.reader(DefaultFeatureResults.java:147)
at org.geotools.gml.producer.FeatureTransformer$FeatureTranslator.encode(FeatureTransformer.java:396)
... 27 more
Caused by: org.geotools.data.DataSourceException: Can't obtain a connection: The maximun of 8 to org.geotools.data.arcsde.ConnectionConfig[dbtype=arcsde, server=*****, port=*****, instance=*****, user=*****, password=*****, minConnections=2, maxConnections=8, connTimeOut=10000, connIncrement=1] has been reached
at org.geotools.data.arcsde.ArcSDEQuery.getConnection(ArcSDEQuery.java:591)
at org.geotools.data.arcsde.ArcSDEQuery.getSeQuery(ArcSDEQuery.java:268)
at org.geotools.data.arcsde.ArcSDEQuery.execute(ArcSDEQuery.java:643)
at org.geotools.data.arcsde.ArcSDEDataStore.getFeatureReader(ArcSDEDataStore.java:567)
... 30 more
Caused by: org.geotools.data.arcsde.UnavailableConnectionException: The maximun of 8 to org.geotools.data.arcsde.ConnectionConfig[dbtype=arcsde, server=*****, port=*****, instance=*****, user=*****, password=*****, minConnections=2, maxConnections=8, connTimeOut=10000, connIncrement=1] has been reached
at org.geotools.data.arcsde.ArcSDEConnectionPool.getConnection(ArcSDEConnectionPool.java:427)
at org.geotools.data.arcsde.ArcSDEConnectionPool.getConnection(ArcSDEConnectionPool.java:367)
at org.geotools.data.arcsde.ArcSDEQuery.getConnection(ArcSDEQuery.java:589)
... 33 more

Farber_Saul_EEA · July 10, 2007, 6:44pm

Brock,

It depends on when this error is happening.

Situation 1 - this happens under heavy load (using a tiled map viewer, for example). This is actually quite likely, as asking for 16 tiles simultaneously from geoserver might cause it to simply use more than 8 connections to ArcSDE, and the later threads probably just quit after waiting for a while. Solution: increase your ArcSDE pool size (from 8 to 20, maybe?) Keep an eye on it and see if this is a 'peak' problem or if it happens consistently.

Situation 2 - this happens after geoserver has been running for 'a while' (anywhere from 5 minutes to 2 months, but it'll be a decently consistent amount of use before the error occurs). There might be a bug in the (now very old) geoserver/geotools codebase that you're using. Probably some bit of code takes control of an ArcSDE connection and then doesn't ever return it to the pool, causing the connection pool to slowly get 'drained' and eventually just not have any connections to give to requests as they come in.

Solution to the second situation is harder. You can increase the pool size (as in situation 1) but this will just delay the problem. You could do something like increase the pool size to something that causes the problem to happen less than once a day, and then have a script restart geoserver every day at some fixed time...this should free up all the connections and reset the pool.

Probably the best solution is to upgrade to the newest GeoServer, but if that's not an option you'll need to improvise something to make the older, now-unsupported code work for you!

If you're a java developer, you could always try and figure out what's going on, too. The code is still around as the geotools 2.1.x branch (or was gs-1.3.x on 2.2.x?)

--saul

________________________________

From: geoserver-users-bounces@lists.sourceforge.net on behalf of Brock Anderson
Sent: Tue 7/10/2007 2:19 PM
To: GeoServer Users
Subject: [Geoserver-users] ArcSDE Connection Pooling

Hi All,

I'm seeing the following error in my Geoserver log file. Looks like
Geoserver tries to go beyond the maximum number of ArcSDE connections
it's configured for. Any thoughts on why that might be?

Some details about my setup:
-Geoserver 1.3.2
-Tomcat
-Windows 2003
-ArcSDE 9.1

I know there are newer versions of Geoserver now, but it's a significant
effort for us to merely upgrade. I wonder if this is a known problem,
and whether there might be an easier fix. Is an upgrade likely to help?

Brock

Caused by: org.geotools.data.DataSourceException: Problem with feature
reader: Can't obtain a connection: The maximun of 8 to
org.geotools.data.arcsde.ConnectionConfig[dbtype=arcsde, server=*****,
port=*****, instance=*****, user=*****, password=*****,
minConnections=2, maxConnections=8, connTimeOut=10000, connIncrement=1]
has been reached
at
org.geotools.data.arcsde.ArcSDEDataStore.getFeatureReader(ArcSDEDataStore.java:598)
at
org.geotools.data.arcsde.ArcSDEDataStore.getFeatureReader(ArcSDEDataStore.java:612)
at
org.geotools.data.DefaultFeatureResults.reader(DefaultFeatureResults.java:147)
at
org.geotools.gml.producer.FeatureTransformer$FeatureTranslator.encode(FeatureTransformer.java:396)
... 27 more
Caused by: org.geotools.data.DataSourceException: Can't obtain a
connection: The maximun of 8 to
org.geotools.data.arcsde.ConnectionConfig[dbtype=arcsde, server=*****,
port=*****, instance=*****, user=*****, password=*****,
minConnections=2, maxConnections=8, connTimeOut=10000, connIncrement=1]
has been reached
at org.geotools.data.arcsde.ArcSDEQuery.getConnection(ArcSDEQuery.java:591)
at org.geotools.data.arcsde.ArcSDEQuery.getSeQuery(ArcSDEQuery.java:268)
at org.geotools.data.arcsde.ArcSDEQuery.execute(ArcSDEQuery.java:643)
at
org.geotools.data.arcsde.ArcSDEDataStore.getFeatureReader(ArcSDEDataStore.java:567)
... 30 more
Caused by: org.geotools.data.arcsde.UnavailableConnectionException: The
maximun of 8 to org.geotools.data.arcsde.ConnectionConfig[dbtype=arcsde,
server=*****, port=*****, instance=*****, user=*****, password=*****,
minConnections=2, maxConnections=8, connTimeOut=10000, connIncrement=1]
has been reached
at
org.geotools.data.arcsde.ArcSDEConnectionPool.getConnection(ArcSDEConnectionPool.java:427)
at
org.geotools.data.arcsde.ArcSDEConnectionPool.getConnection(ArcSDEConnectionPool.java:367)
at org.geotools.data.arcsde.ArcSDEQuery.getConnection(ArcSDEQuery.java:589)
... 33 more

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Brock_Anderson · July 11, 2007, 7:21pm

Saul,

Your 'situation 2' seems likely. I'm using Geoserver as a WFS only, and it's not generally bombarded with requests.

Does anyone have reason to believe that this issue was likely resolved since Geoserver 1.3.2?

Brock

Farber, Saul (EEA) wrote:

Brock,
It depends on when this error is happening.
Situation 1 - this happens under heavy load (using a tiled map viewer, for example). This is actually quite likely, as asking for 16 tiles simultaneously from geoserver might cause it to simply use more than 8 connections to ArcSDE, and the later threads probably just quit after waiting for a while. Solution: increase your ArcSDE pool size (from 8 to 20, maybe?) Keep an eye on it and see if this is a 'peak' problem or if it happens consistently.
Situation 2 - this happens after geoserver has been running for 'a while' (anywhere from 5 minutes to 2 months, but it'll be a decently consistent amount of use before the error occurs). There might be a bug in the (now very old) geoserver/geotools codebase that you're using. Probably some bit of code takes control of an ArcSDE connection and then doesn't ever return it to the pool, causing the connection pool to slowly get 'drained' and eventually just not have any connections to give to requests as they come in.
Solution to the second situation is harder. You can increase the pool size (as in situation 1) but this will just delay the problem. You could do something like increase the pool size to something that causes the problem to happen less than once a day, and then have a script restart geoserver every day at some fixed time...this should free up all the connections and reset the pool.

If you're a java developer, you could always try and figure out what's going on, too. The code is still around as the geotools 2.1.x branch (or was gs-1.3.x on 2.2.x?)
--saul

________________________________

From: geoserver-users-bounces@lists.sourceforge.net on behalf of Brock Anderson
Sent: Tue 7/10/2007 2:19 PM
To: GeoServer Users
Subject: [Geoserver-users] ArcSDE Connection Pooling

Hi All,

I'm seeing the following error in my Geoserver log file. Looks like
Geoserver tries to go beyond the maximum number of ArcSDE connections
it's configured for. Any thoughts on why that might be?

Some details about my setup:
-Geoserver 1.3.2
-Tomcat
-Windows 2003
-ArcSDE 9.1

I know there are newer versions of Geoserver now, but it's a significant
effort for us to merely upgrade. I wonder if this is a known problem,
and whether there might be an easier fix. Is an upgrade likely to help?

Brock

Caused by: org.geotools.data.DataSourceException: Problem with feature
reader: Can't obtain a connection: The maximun of 8 to
org.geotools.data.arcsde.ConnectionConfig[dbtype=arcsde, server=*****,
port=*****, instance=*****, user=*****, password=*****,
minConnections=2, maxConnections=8, connTimeOut=10000, connIncrement=1]
has been reached
at
org.geotools.data.arcsde.ArcSDEDataStore.getFeatureReader(ArcSDEDataStore.java:598)
at
org.geotools.data.arcsde.ArcSDEDataStore.getFeatureReader(ArcSDEDataStore.java:612)
at
org.geotools.data.DefaultFeatureResults.reader(DefaultFeatureResults.java:147)
at
org.geotools.gml.producer.FeatureTransformer$FeatureTranslator.encode(FeatureTransformer.java:396)
... 27 more
Caused by: org.geotools.data.DataSourceException: Can't obtain a
connection: The maximun of 8 to
org.geotools.data.arcsde.ConnectionConfig[dbtype=arcsde, server=*****,
port=*****, instance=*****, user=*****, password=*****,
minConnections=2, maxConnections=8, connTimeOut=10000, connIncrement=1]
has been reached
at org.geotools.data.arcsde.ArcSDEQuery.getConnection(ArcSDEQuery.java:591)
at org.geotools.data.arcsde.ArcSDEQuery.getSeQuery(ArcSDEQuery.java:268)
at org.geotools.data.arcsde.ArcSDEQuery.execute(ArcSDEQuery.java:643)
at
org.geotools.data.arcsde.ArcSDEDataStore.getFeatureReader(ArcSDEDataStore.java:567)
... 30 more
Caused by: org.geotools.data.arcsde.UnavailableConnectionException: The
maximun of 8 to org.geotools.data.arcsde.ConnectionConfig[dbtype=arcsde,
server=*****, port=*****, instance=*****, user=*****, password=*****,
minConnections=2, maxConnections=8, connTimeOut=10000, connIncrement=1]
has been reached
at
org.geotools.data.arcsde.ArcSDEConnectionPool.getConnection(ArcSDEConnectionPool.java:427)
at
org.geotools.data.arcsde.ArcSDEConnectionPool.getConnection(ArcSDEConnectionPool.java:367)
at org.geotools.data.arcsde.ArcSDEQuery.getConnection(ArcSDEQuery.java:589)
... 33 more

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Justin_Deoliveira · July 11, 2007, 8:19pm

It is very likely. GeoServer itself has made huge strides since 1.3.x, the current stable release is 1.5.1. And especially with arcsde since Saul took over as module maintainer for it.

-Justin

Brock Anderson wrote:

Saul,

Your 'situation 2' seems likely. I'm using Geoserver as a WFS only, and it's not generally bombarded with requests.

Does anyone have reason to believe that this issue was likely resolved since Geoserver 1.3.2?

Brock

Farber, Saul (EEA) wrote:

Brock,
It depends on when this error is happening.
Situation 1 - this happens under heavy load (using a tiled map viewer, for example). This is actually quite likely, as asking for 16 tiles simultaneously from geoserver might cause it to simply use more than 8 connections to ArcSDE, and the later threads probably just quit after waiting for a while. Solution: increase your ArcSDE pool size (from 8 to 20, maybe?) Keep an eye on it and see if this is a 'peak' problem or if it happens consistently.
Situation 2 - this happens after geoserver has been running for 'a while' (anywhere from 5 minutes to 2 months, but it'll be a decently consistent amount of use before the error occurs). There might be a bug in the (now very old) geoserver/geotools codebase that you're using. Probably some bit of code takes control of an ArcSDE connection and then doesn't ever return it to the pool, causing the connection pool to slowly get 'drained' and eventually just not have any connections to give to requests as they come in.
Solution to the second situation is harder. You can increase the pool size (as in situation 1) but this will just delay the problem. You could do something like increase the pool size to something that causes the problem to happen less than once a day, and then have a script restart geoserver every day at some fixed time...this should free up all the connections and reset the pool.

If you're a java developer, you could always try and figure out what's going on, too. The code is still around as the geotools 2.1.x branch (or was gs-1.3.x on 2.2.x?)
--saul

________________________________

From: geoserver-users-bounces@lists.sourceforge.net on behalf of Brock Anderson
Sent: Tue 7/10/2007 2:19 PM
To: GeoServer Users
Subject: [Geoserver-users] ArcSDE Connection Pooling

Hi All,

I'm seeing the following error in my Geoserver log file. Looks like
Geoserver tries to go beyond the maximum number of ArcSDE connections
it's configured for. Any thoughts on why that might be?

Some details about my setup:
-Geoserver 1.3.2
-Tomcat
-Windows 2003
-ArcSDE 9.1

I know there are newer versions of Geoserver now, but it's a significant
effort for us to merely upgrade. I wonder if this is a known problem,
and whether there might be an easier fix. Is an upgrade likely to help?

Brock

Caused by: org.geotools.data.DataSourceException: Problem with feature
reader: Can't obtain a connection: The maximun of 8 to
org.geotools.data.arcsde.ConnectionConfig[dbtype=arcsde, server=*****,
port=*****, instance=*****, user=*****, password=*****,
minConnections=2, maxConnections=8, connTimeOut=10000, connIncrement=1]
has been reached
at
org.geotools.data.arcsde.ArcSDEDataStore.getFeatureReader(ArcSDEDataStore.java:598)
at
org.geotools.data.arcsde.ArcSDEDataStore.getFeatureReader(ArcSDEDataStore.java:612)
at
org.geotools.data.DefaultFeatureResults.reader(DefaultFeatureResults.java:147)
at
org.geotools.gml.producer.FeatureTransformer$FeatureTranslator.encode(FeatureTransformer.java:396)
... 27 more
Caused by: org.geotools.data.DataSourceException: Can't obtain a
connection: The maximun of 8 to
org.geotools.data.arcsde.ConnectionConfig[dbtype=arcsde, server=*****,
port=*****, instance=*****, user=*****, password=*****,
minConnections=2, maxConnections=8, connTimeOut=10000, connIncrement=1]
has been reached
at org.geotools.data.arcsde.ArcSDEQuery.getConnection(ArcSDEQuery.java:591)
at org.geotools.data.arcsde.ArcSDEQuery.getSeQuery(ArcSDEQuery.java:268)
at org.geotools.data.arcsde.ArcSDEQuery.execute(ArcSDEQuery.java:643)
at
org.geotools.data.arcsde.ArcSDEDataStore.getFeatureReader(ArcSDEDataStore.java:567)
... 30 more
Caused by: org.geotools.data.arcsde.UnavailableConnectionException: The
maximun of 8 to org.geotools.data.arcsde.ConnectionConfig[dbtype=arcsde,
server=*****, port=*****, instance=*****, user=*****, password=*****,
minConnections=2, maxConnections=8, connTimeOut=10000, connIncrement=1]
has been reached
at
org.geotools.data.arcsde.ArcSDEConnectionPool.getConnection(ArcSDEConnectionPool.java:427)
at
org.geotools.data.arcsde.ArcSDEConnectionPool.getConnection(ArcSDEConnectionPool.java:367)
at org.geotools.data.arcsde.ArcSDEQuery.getConnection(ArcSDEQuery.java:589)
... 33 more

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:4007,46952da236802458217002!

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org