[Geoserver-users] ArcSDE connection pooling issues

Bart_van_den_Eijnden · February 20, 2008, 4:01pm

Some more info, WMS keeps on working fine, whereas WFS-T support breaks
after a few (about 10) transactions, even after increasing the timeout to
10000.

I also get a lot of broken pipe errors:

2008-02-20 16:59:58,034 INFO [org.geotools.arcsde.pool.ArcSDEConnectionPool]
- Can't validate SeConnection, discarding it:
org.geotools.arcsde.pool.ArcSDEPooledConnection@anonymised.com
2008-02-20 16:59:58,035 INFO [org.geotools.arcsde.pool] - closing
connection: Broken pipe
2008-02-20 16:59:58,036 WARN [org.geoserver.ows] -
java.lang.IndexOutOfBoundsException
        at
java.io.ByteArrayInputStream.read(ByteArrayInputStream.java(Compiled Code))
        at java.io.DataInputStream.read(DataInputStream.java(Inlined
Compiled Code))
        at com.esri.sde.sdk.client.r$a_.a(Unknown Source)
        at com.esri.sde.sdk.client.r$a_.l(Unknown Source)
        at com.esri.sde.sdk.client.r$a_.b(Unknown Source)
        at com.esri.sde.sdk.client.r$a_.b(Unknown Source)
        at com.esri.sde.sdk.client.r.j(Unknown Source)
        at com.esri.sde.sdk.client.b.h(Unknown Source)
        at com.esri.sde.sdk.client.b.o(Unknown Source)
        at com.esri.sde.sdk.client.j.a(Unknown Source)
        at com.esri.sde.sdk.client.SeQuery.calculateTableStatistics(Unknown
Source)
        at
org.geotools.arcsde.data.ArcSDEQuery.calculateResultCount(ArcSDEQuery.java:632)
        at
org.geotools.arcsde.data.ArcSDEQuery.calculateResultCount(ArcSDEQuery.java:543)
        at
org.geotools.arcsde.data.ArcSDEDataStore.getCount(ArcSDEDataStore.java:996)
        at
org.geotools.data.AbstractFeatureSource.getCount(AbstractFeatureSource.java:310)
        at
org.geotools.data.DefaultFeatureResults.getCount(DefaultFeatureResults.java:295)
        at
org.geotools.data.store.DataFeatureCollection.size(DataFeatureCollection.java:282)
        at
org.geotools.data.crs.ForceCoordinateSystemFeatureResults.size(ForceCoordinateSystemFeatureResults.java:122)
        at org.geoserver.wfs.GetFeature.run(GetFeature.java:254)
        at
org.geoserver.wfs.DefaultWebFeatureService.getFeature(DefaultWebFeatureService.java:108)
        at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java(Compiled
Code))
        at java.lang.reflect.Method.invoke(Method.java(Compiled Code))
        at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:292)
        at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:155)

Best regards,
Bart

--
Bart van den Eijnden
OSGIS, Open Source GIS
http://www.osgis.nl

--------- Oorspronkelijk bericht --------
Van: Bart van den Eijnden OSGIS <bartvde@anonymised.com>
Naar: Gabriel Roldán <groldan@anonymised.com>,
geoserver-users@lists.sourceforge.net
<geoserver-users@lists.sourceforge.net>, Bart van den Eijnden OSGIS
<bartvde@anonymised.com>
Cc: Saul Farber <Saul.Farber@anonymised.com>
Onderwerp: Re: [Geoserver-users] ArcSDE connection pooling issues
Datum: 20/02/08 13:49

Hi Gabriel,

the OL layers are WFS layers, so Untiled. There are only 3 layers (one for
points, polygons and lines) in the application, and using OpenLayers
(Layer.WFS) new features are added.

So we don't use WMS, only WFS in this case.

I have the default value for the connection timeout, I'll try increasing

it.

Best regards,
Bart

--
Bart van den Eijnden
OSGIS, Open Source GIS
http://www.osgis.nl

--------- Oorspronkelijk bericht --------
Van: Gabriel Roldán <groldan@anonymised.com>
Naar: geoserver-users@lists.sourceforge.net
<geoserver-users@lists.sourceforge.net>, Bart van den Eijnden OSGIS
<bartvde@anonymised.com>
Cc: Saul Farber <Saul.Farber@anonymised.com>
Onderwerp: Re: [Geoserver-users] ArcSDE connection pooling issues
Datum: 20/02/08 13:41

> Hi Bart,
>
> My first thought is it may be a matter of balance bettween connection

pool

> size and pool's connTimeout argument. Did you try increasing the
connTimeout
> parameter in the DataStore config?
>
> question though: is the openlayers client requesting tiles? I've to

admit

I
> never tried a tiling client over arcsde layers myself, but it would

make

> sense to run out of connections depending on the ammount of tiles

being

> requested at a time and the ammount of layers composing the map.
> Yet its though. I've been trying Saul's massgis WMS[1] through uDig

before

> replying to this message and it seems to perform well even requesting

20

> layers at a time.
>
> May be Saul (cc'ed) can give us a couple clues about how to set up a
> production geoserver backed up by arcsde?
>
> Saul: did you ever run into connection exhaustion issues with your

server

and,
> if so, how did you solved them? is it just a matter of having a ton

of

> available connections? and did you ever tried hitting it with a

tiling

> client?
>
> cheers,
>
> Gabriel
>
> [1]Massgis WMS server:
>

&lt;http://giswebservices.massgis.state.ma.us/geoserver/wms?service=WMS&amp;request=GetCapabilities&gt;

>
> On Wednesday 20 February 2008 02:20:39 pm Bart van den Eijnden

(OSGIS)

wrote:
> &gt; Hi list,
> &gt;
> &gt; we are running into connection pooling issues (using

1.6RC2), we now

have
> &gt; even increased the maximum number up to 25 (coming from 8),

but still

> &gt; Geoserver runs out of connections after a while. There is

only 1

client
> &gt; (OpenLayers based) active!
> &gt;
> &gt; What could be the issue?
> &gt;
> &gt; Caused by: java.util.NoSuchElementException: Could not

aquire

> &gt; feature:org.geotools.data.DataSourceException: The maximun

of 25 to

> &gt;

org.geotools.arcsde.pool.ArcSDEConnectionConfig[dbtype=arcsde,

> &gt; server=geodatabase.ad.rws.nl, port=5151, instance=null,

user=DID,

> &gt; password=*****, minConnections=2, maxConnections=25,
connTimeOut=1000] has
> &gt; been reached
> &gt;
> &gt; 2008-02-20 11:17:03,298 WARN
> &gt; [org.geotools.arcsde.pool.ArcSDEConnectionPool] - Out of

connections:

> &gt; Timeout waiting for idle object
> &gt; java.util.NoSuchElementException: Timeout waiting for idle

object

> &gt; at
> &gt;

org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPo

> &gt;ol.java(Compiled Code))
> &gt; at
> &gt;

org.geotools.arcsde.pool.ArcSDEConnectionPool.getConnection(ArcSDEConnectio

> &gt;nPool.java(Compiled Code))
> &gt; at
> &gt;

org.geotools.arcsde.data.ArcSDEDataStore.getFeatureWriterAppend(ArcSDEDataS

> &gt;tore.java:963) at
> &gt;

org.geotools.data.AbstractFeatureStore.addFeatures(AbstractFeatureStore.jav

> &gt;a:249) at
> &gt;

org.vfny.geoserver.global.GeoServerFeatureStore.addFeatures(GeoServerFeatur

> &gt;eStore.java:76) at
> &gt;

org.geoserver.wfs.InsertElementHandler.execute(InsertElementHandler.java:16

> &gt;4) at

org.geoserver.wfs.Transaction.execute(Transaction.java:328) at

> &gt;

org.geoserver.wfs.Transaction.transaction(Transaction.java:104) at

> &gt;

org.geoserver.wfs.DefaultWebFeatureService.transaction(DefaultWebFeatureSer

> &gt;vice.java:156) at
sun.reflect.GeneratedMethodAccessor203.invoke(Unknown
> &gt; Source) at
> &gt;

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp

> &gt;l.java(Compiled Code))
> &gt; at

java.lang.reflect.Method.invoke(Method.java(Compiled

Code))
> &gt; at
> &gt;

org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(Aop

> &gt;Utils.java:292) at
> &gt;

org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoin

> &gt;t(ReflectiveMethodInvocation.java:155) at
> &gt;

org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Reflec

> &gt;tiveMethodInvocation.java:122) at
> &gt; org.geoserver.wfs.WFSLogger.invoke(WFSLogger.java:44)
> &gt; at
> &gt;

org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Reflec

> &gt;tiveMethodInvocation.java:144) at
> &gt;

org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopPr

> &gt;oxy.java:174) at $Proxy0.transaction(Unknown Source)
> &gt; at

sun.reflect.GeneratedMethodAccessor306.invoke(Unknown

Source)
> &gt; at
> &gt;

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp

> &gt;l.java(Compiled Code))
> &gt; at

java.lang.reflect.Method.invoke(Method.java(Compiled

Code))
> &gt; at
> &gt;

org.geoserver.security.OperationSecurityInterceptor.invoke(OperationSecurit

> &gt;yInterceptor.java:41) at
> &gt; org.geoserver.ows.Dispatcher.execute(Dispatcher.java:495) at
> &gt;
org.geoserver.ows.Dispatcher.handleRequestInternal(Dispatcher.java:188)
> &gt; at
> &gt;

org.springframework.web.servlet.mvc.AbstractController.handleRequest(Abstra

> &gt;ctController.java:139) at
> &gt;

org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(S

> &gt;impleControllerHandlerAdapter.java:44) at
> &gt;

org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServ

> &gt;let.java:684) at
> &gt;

org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServl

> &gt;et.java:625) at
> &gt;

org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkSe

> &gt;rvlet.java:392) at
> &gt;

org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.ja

> &gt;va:357) at
javax.servlet.http.HttpServlet.service(HttpServlet.java:709) at
> &gt;

javax.servlet.http.HttpServlet.service(HttpServlet.java(Compiled

Code))
> &gt; at
> &gt;

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicatio

> &gt;nFilterChain.java(Compiled Code))
> &gt; at
> &gt;

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterC

> &gt;hain.java(Compiled Code))
> &gt; at
> &gt;

org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacter

> &gt;EncodingFilter.java:108) at
> &gt;

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicatio

> &gt;nFilterChain.java(Compiled Code))
> &gt; at
> &gt;

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterC

> &gt;hain.java(Compiled Code))
> &gt; at
> &gt;