[GeoNetwork-users] Geonetwork and JNDI connection pooling

I have been testing out GN 2.4.2 on a live Tomcat 6.0.18 server (Linux) for the past few weeks, and have noticed a problem which arises as a result of the application being idle longer than a certain time. I understand why it happens (it is to do with stale Postgres database connections in a pool which the app thinks are still alive, but which our firewall thinks are not). The result of this is that any access to GN hangs (the logs report that the correct request was sent, just that nothing ever returns). There are reports from the JDBC backend of timeouts and "NoRouteToHostException". The relevant portion of my GN config.xml is:

<resource enabled="true">
        <name>main-db</name>
        <provider>jeeves.resources.dbms.DbmsPool</provider>
        <config>
                <user>username</user>
                <password>password</password>
                <driver>org.postgresql.Driver</driver>
                <url>jdbc:postgresql://servername:5432/geonetwork</url>
                <poolSize>10</poolSize>
        </config>
</resource>

Is there a way round this using an extra config parameters in the config.xml file, e.g. to validate connections in the pool before attempting to use them? I'd appreciate a pointer as to where to look to find out what other parameters are available for config. It seems from Jeeves docs that I *could* change the provider over (e.g. to javax.sql.DataSource?). The thinking behind this is that I could use a JNDI resource defined in the Tomcat application Context e.g:

<Resource name="jdbc/Geonetwork"
        type="javax.sql.DataSource"
        initialSize="20"
        maxActive="100"
        maxIdle="4"
        minIdle="0"
        maxWait="60000"
        validationQuery="SELECT 1"
        removeAbandoned="true"
        removeAbandonedTimeout="60"
        password="password"
        driverClassName="org.postgresql.Driver"
        username="username"
        url="jdbc:postgresql://servername:5432/geonetwork"
        />

This allows for connection validation and discarding of invalid ones. I would like to know if anyone has got GN to run with JNDI database connection pooling rather than the Jeeves default (or whether it's even possible!). Is this a realistic possibility, or am I in the business of creating my own custom version of the source?

With thanks,

David Herbert
British Antarctic Survey.

Geonetwork gurus,

I wanted to second David's request. We have seen the same problem here with firewalls killing connections that Jeeves thinks are still alive.

The Tomcat catalina.out log file shows messages like:

reconnecting: 322557763>=1 ms since last connection
java.sql.SQLRecoverableException: Io exception: Connection reset
...
ClientAbortException: java.net.SocketException: Broken pipe
net.sf.saxon.trans.XPathException: ClientAbortException: java.net.SocketException: Broken pipe

And geonetwork.log:

2010-03-12 11:25:10,388 DEBUG [jeeves.service] - Raised exception while executing service
<error id="error">
<message />
<class>ClientAbortException</class>
<stack>
<at class="org.apache.catalina.connector.OutputBuffer" file="OutputBuffer.java" line="366" method="realWriteBytes" />

We run Tomcat 5.5 on Red Hat Linux RHEL5, connecting to an Oracle 10.2 database.

We'd really appreciate any ideas on how to avoid these errors!

Thanks very much,
--Rich

--

Richard Fozzard, Computer Scientist
  Geospatial Metadata at NGDC: http://www.ngdc.noaa.gov/metadata

Cooperative Institute for Research in Environmental Sciences (CIRES)
Univ. Colorado & NOAA National Geophysical Data Center, Enterprise Data Systems 325 S. Broadway, Skaggs 1B-305, Boulder, CO 80305
Office: 303-497-6487, Cell: 303-579-5615, Email: richard.fozzard@anonymised.com

Herbert, David J said the following on 03/17/2010 08:31 AM:

I have been testing out GN 2.4.2 on a live Tomcat 6.0.18 server (Linux) for the past few weeks, and have noticed a problem which arises as a result of the application being idle longer than a certain time. I understand why it happens (it is to do with stale Postgres database connections in a pool which the app thinks are still alive, but which our firewall thinks are not). The result of this is that any access to GN hangs (the logs report that the correct request was sent, just that nothing ever returns). There are reports from the JDBC backend of timeouts and "NoRouteToHostException". The relevant portion of my GN config.xml is:

<resource enabled="true">
        <name>main-db</name>
        <provider>jeeves.resources.dbms.DbmsPool</provider>
        <config>
                <user>username</user>
                <password>password</password>
                <driver>org.postgresql.Driver</driver>
                <url>jdbc:postgresql://servername:5432/geonetwork</url>
                <poolSize>10</poolSize>
        </config>
</resource>

Is there a way round this using an extra config parameters in the config.xml file, e.g. to validate connections in the pool before attempting to use them? I'd appreciate a pointer as to where to look to find out what other parameters are available for config. It seems from Jeeves docs that I *could* change the provider over (e.g. to javax.sql.DataSource?). The thinking behind this is that I could use a JNDI resource defined in the Tomcat application Context e.g:

<Resource name="jdbc/Geonetwork" type="javax.sql.DataSource"
        initialSize="20"
        maxActive="100"
        maxIdle="4"
        minIdle="0"
        maxWait="60000"
        validationQuery="SELECT 1"
        removeAbandoned="true"
        removeAbandonedTimeout="60" password="password" driverClassName="org.postgresql.Driver"
        username="username" url="jdbc:postgresql://servername:5432/geonetwork" />

This allows for connection validation and discarding of invalid ones. I would like to know if anyone has got GN to run with JNDI database connection pooling rather than the Jeeves default (or whether it's even possible!). Is this a realistic possibility, or am I in the business of creating my own custom version of the source?

With thanks,

David Herbert
British Antarctic Survey.

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
  

I'll third that request. (If a third is needed. {grin})

Moving connection pooling out of the web-app into the JEE container is good architecture.

---
A. Soroka
Digital Research and Scholarship R & D
the University of Virginia Library

On Mar 17, 2010, at 2:25 PM, Richard Fozzard wrote:

Geonetwork gurus,

I wanted to second David's request. We have seen the same problem here
with firewalls killing connections that Jeeves thinks are still alive.

The Tomcat catalina.out log file shows messages like:

reconnecting: 322557763>=1 ms since last connection
java.sql.SQLRecoverableException: Io exception: Connection reset
...
ClientAbortException: java.net.SocketException: Broken pipe
net.sf.saxon.trans.XPathException: ClientAbortException:
java.net.SocketException: Broken pipe

And geonetwork.log:

2010-03-12 11:25:10,388 DEBUG [jeeves.service] - Raised exception while
executing service
<error id="error">
<message />
<class>ClientAbortException</class>
<stack>
<at class="org.apache.catalina.connector.OutputBuffer"
file="OutputBuffer.java" line="366" method="realWriteBytes" />

We run Tomcat 5.5 on Red Hat Linux RHEL5, connecting to an Oracle 10.2
database.

We'd really appreciate any ideas on how to avoid these errors!

Thanks very much,
--Rich

--

Richard Fozzard, Computer Scientist
Geospatial Metadata at NGDC: http://www.ngdc.noaa.gov/metadata

Cooperative Institute for Research in Environmental Sciences (CIRES)
Univ. Colorado & NOAA National Geophysical Data Center, Enterprise Data Systems
325 S. Broadway, Skaggs 1B-305, Boulder, CO 80305
Office: 303-497-6487, Cell: 303-579-5615, Email: richard.fozzard@anonymised.com

Herbert, David J said the following on 03/17/2010 08:31 AM:

I have been testing out GN 2.4.2 on a live Tomcat 6.0.18 server (Linux) for the past few weeks, and have noticed a problem which arises as a result of the application being idle longer than a certain time. I understand why it happens (it is to do with stale Postgres database connections in a pool which the app thinks are still alive, but which our firewall thinks are not). The result of this is that any access to GN hangs (the logs report that the correct request was sent, just that nothing ever returns). There are reports from the JDBC backend of timeouts and "NoRouteToHostException". The relevant portion of my GN config.xml is:

<resource enabled="true">
       <name>main-db</name>
       <provider>jeeves.resources.dbms.DbmsPool</provider>
       <config>
               <user>username</user>
               <password>password</password>
               <driver>org.postgresql.Driver</driver>
               <url>jdbc:postgresql://servername:5432/geonetwork</url>
               <poolSize>10</poolSize>
       </config>
</resource>

Is there a way round this using an extra config parameters in the config.xml file, e.g. to validate connections in the pool before attempting to use them? I'd appreciate a pointer as to where to look to find out what other parameters are available for config. It seems from Jeeves docs that I *could* change the provider over (e.g. to javax.sql.DataSource?). The thinking behind this is that I could use a JNDI resource defined in the Tomcat application Context e.g:

<Resource name="jdbc/Geonetwork"
        type="javax.sql.DataSource"
        initialSize="20"
        maxActive="100"
        maxIdle="4"
        minIdle="0"
        maxWait="60000"
        validationQuery="SELECT 1"
        removeAbandoned="true"
        removeAbandonedTimeout="60"
        password="password"
        driverClassName="org.postgresql.Driver"
        username="username"
        url="jdbc:postgresql://servername:5432/geonetwork"
        />

This allows for connection validation and discarding of invalid ones. I would like to know if anyone has got GN to run with JNDI database connection pooling rather than the Jeeves default (or whether it's even possible!). Is this a realistic possibility, or am I in the business of creating my own custom version of the source?

With thanks,

David Herbert
British Antarctic Survey.

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork