[GeoNetwork-devel] [GeoNetwork opensource Developer website] #165: "unable to open resource main-db" returned by metadata.show and xml.iso19139 services - stalls server?

#165: "unable to open resource main-db" returned by metadata.show and
xml.iso19139 services - stalls server?
---------------------+------------------------------------------------------
Reporter: simonp | Owner: geonetwork-devel@lists.sourceforge.net
     Type: defect | Status: new
Priority: major | Milestone: v2.4.1
Component: General | Version: v2.5.0
Keywords: |
---------------------+------------------------------------------------------
Reported by Heikki Doeleman:

________________________________________
From: James Wilson [james.q.wilson@...275...]
Sent: Wednesday, 21 October 2009 3:02 AM
To: geonetwork-devel@lists.sourceforge.net
Subject: Re: [GeoNetwork-devel] "unable to open resource main-db"

Heikki,

I'm also seeing this problem after doing repeated saves. I'm using oracle,
with pool size = 10 and reconnectTime=undefined. Problem is not easily
repeatable, but happens regularly.

Is this a problem with the connection pool running out of objects?

Reading the code, I'm somewhat confused as to how the connection pool is
supposed to work, and resources get released. By my reading, the only
place
connections get released is by calling close on connection pool object
(DbmsPool) with the resource (Dbms) as an argument, which unlocks the
resource.

However, in the code, this never seems to be done. The default pattern
seems
to be to get a new resource with

dbms = context.getResourceManager().open(Geonet.Res.MAIN_DB)

and then to leave the resource to go out of scope

Yours, mystified, and frustrated at not being able to track down the
error....

Anybody else got any ideas?

James

Heikki Doeleman wrote:
>
> hello lists,
>
> I'm seeing this error, every so often :
>
> **"unable to open resource main-db after 20attempts"
>
> This happens when you're using the editor for a bit. When it happens,
you
> must either restart GeoNetwork or wait a while (session timeout?).
>
> It started to happen for us after moving to Postgres and moving to a GN
> 2.4-based application.
>
> A search on Google shows that quite lots of GeoNetwork nodes have it.
> Moreover, from a WHO node that displays a stacktrace, it seems they have
> it
> when using MySQL, not Postgres (
>
http://209.85.229.132/search?q=cache:5Z0sxBf6fS8J:apps.who.int/geonetwork/srv/en/metadata.show%3Fid%3D74%26currTab%3Dsimple+"unable+to+open+resource+
main-
db%22+%22Communications+link+failure+due+to+underlying+exception:%22&cd=1&hl=en&ct=clnk
> ).
>
> Does anybody know what to do to prevent this from ever happening ? My
JDBC
> settings in config.xml are poolSize=10, reconnectTime=undefined
> (defaulting
> to Jeeves).
>
>
> Thank you so much
>
> Kind regards
> Heikki Doeleman
>

--
Ticket URL: <http://trac.osgeo.org/geonetwork/ticket/165&gt;
GeoNetwork opensource Developer website <http://trac.osgeo.org/geonetwork&gt;
GeoNetwork opensource is a standards based, Free and Open Source catalog application to manage spatially referenced resources through the web. It provides powerful metadata editing and search functions as well as an embedded interactive web map viewer. This website contains information related to the development of the software.

#165: "unable to open resource main-db" returned by metadata.show and
xml.iso19139 services - stalls server?
----------------------+-----------------------------------------------------
  Reporter: simonp | Owner: geonetwork-devel@lists.sourceforge.net
      Type: defect | Status: new
  Priority: major | Milestone: v2.4.1
Component: General | Version: v2.5.0
Resolution: | Keywords:
----------------------+-----------------------------------------------------
Comment (by simonp):

Looking at Heikki's original message and references to stack traces in
google - its actually taking place in the services metadata.show and
iso19139.xml. I can reproduce it with curl and thumping the server with
lots of requests to the metadata.show service.

Seems that some code, somewhere was opening a Dbms resource but wasn't
going through the usual Jeeves clean up of Dbms resources (via commit on
success or rollback on exception) at the end of a service, hence the 10
Dbms connections in the pool would eventually end up locked and an
exception with message 'unable to open main-db after 20 attempts' would be
given. The code appears to be the timertask thread that is used to
increase popularity asynchronously in metadata.show (its also called by
iso19139.xml). One fix is to add:

context.getResourceManager().close();

as the last line of the try block in the run method of the
IncreasePopularityTask class in
src/org/fao/geonet/services/metadata/Show.java

I'm not sure this is good though as when thumping the server (albeit with
a somewhat unrealistic load) as described above, the thread that commits
the increase in popularity tends to overlap with other threads that
attempt to select from the Metadata table - the select threads fail
because of pending commits. Safest might be to remove the async popularity
update task and just run dm.increasePopularity(dbms, id); synchronously
(no exceptions there)?

Cheers,
Simon

--
Ticket URL: <http://trac.osgeo.org/geonetwork/ticket/165#comment:1&gt;
GeoNetwork opensource Developer website <http://trac.osgeo.org/geonetwork&gt;
GeoNetwork opensource is a standards based, Free and Open Source catalog application to manage spatially referenced resources through the web. It provides powerful metadata editing and search functions as well as an embedded interactive web map viewer. This website contains information related to the development of the software.

#165: "unable to open resource main-db" returned by metadata.show and
xml.iso19139 services - stalls server?
----------------------+-----------------------------------------------------
  Reporter: simonp | Owner: geonetwork-devel@lists.sourceforge.net
      Type: defect | Status: new
  Priority: major | Milestone: v2.4.1
Component: General | Version: v2.5.0
Resolution: | Keywords:
----------------------+-----------------------------------------------------
Comment (by simonp):

context.getResourceManager().close(); added in commit 5384

--
Ticket URL: <http://trac.osgeo.org/geonetwork/ticket/165#comment:2&gt;
GeoNetwork opensource Developer website <http://trac.osgeo.org/geonetwork&gt;
GeoNetwork opensource is a standards based, Free and Open Source catalog application to manage spatially referenced resources through the web. It provides powerful metadata editing and search functions as well as an embedded interactive web map viewer. This website contains information related to the development of the software.

#165: "unable to open resource main-db" returned by metadata.show and
xml.iso19139 services - stalls server?
----------------------+-----------------------------------------------------
  Reporter: simonp | Owner: geonetwork-devel@lists.sourceforge.net
      Type: defect | Status: closed
  Priority: major | Milestone: v2.4.3
Component: General | Version: v2.5.0
Resolution: fixed | Keywords:
----------------------+-----------------------------------------------------
Changes (by simonp):

  * status: new => closed
  * resolution: => fixed

--
Ticket URL: <http://trac.osgeo.org/geonetwork/ticket/165#comment:3&gt;
GeoNetwork opensource Developer website <http://trac.osgeo.org/geonetwork&gt;
GeoNetwork opensource is a standards based, Free and Open Source catalog application to manage spatially referenced resources through the web. It provides powerful metadata editing and search functions as well as an embedded interactive web map viewer. This website contains information related to the development of the software.

We are investigating the creation of an Amazon Machine Instance (virtual machine installation package) for Linux 64-bit, Java, PostgreSQL, and GeoNetwork. Any suggestions from the developer community on selecting a version of Linux-64 that would provide the best performing and safest base for such a solution? Also, has anyone created an AMI for GeoNetwork yet?

Doug.

--
Douglas D. Nebert
Senior Advisor for Geospatial Technology, System-of-Systems Architect
FGDC Secretariat T:703 648 4151 F:703 648-5755 C:703 459-5860

Hi Doug -

At the Arizona Geological Survey we’ve been using Amazon VMs to provide GeoNetwork and Geoserver services. http://catalog.usgin.org/geonetwork is the current incarnation. Our goal is to try and build something that could be handed off to other state surveys with relative ease. However, we haven’t gotten to the point that we’ve actually tried to hand it off yet. There is some decent documentation of our build process online at lab.usgin.org/applications/doc/development-usgin-amazon-machine-image-ami

To answer your questions more directly, we didn’t go 64-bit, but we are using Ubuntu Server 9.10, started with a machine image provided by http://alestic.com and we’re using a version of GeoNetwork built from the SVN trunk. Alestic.com also provides 64-bit Ubuntu images, and I’d be interested to know how they work if you decide to go that route.

We haven’t created a public AMI based on our build yet, but we’ve been meaning to, and if you’re interested in trying out our build this could be a good opportunity to get us to jumpstart the process. Please get in touch with me if you’ve got any questions.

Ryan

At SpatialCloud.com we have been using AMIs built by Eric Hammond
(alestic.com) for some time now. This includes both our customer
facing instances and ones we use for data processing in the
background. Eric has been at this for some time now and we have good
results working off his builds. His site is also a good place to go
for info.

Ubuntu 10.04 Lucid is expected to be released this Thursday (April 29,
2010). This is the first LTS (long term support) version to be
released since Ubuntu 8.04 Hardy so that might be a good target.

On Mon, Apr 26, 2010 at 3:49 PM, Doug Nebert <ddnebert@anonymised.com> wrote:

We are investigating the creation of an Amazon Machine Instance (virtual
machine installation package) for Linux 64-bit, Java, PostgreSQL, and
GeoNetwork. Any suggestions from the developer community on selecting a
version of Linux-64 that would provide the best performing and safest
base for such a solution? Also, has anyone created an AMI for GeoNetwork
yet?

Doug.

--
Douglas D. Nebert
Senior Advisor for Geospatial Technology, System-of-Systems Architect
FGDC Secretariat T:703 648 4151 F:703 648-5755 C:703 459-5860

------------------------------------------------------------------------------
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Yes, I think they come out on the same day even.

On Tue, Apr 27, 2010 at 9:52 AM, Doug Nebert <ddnebert@anonymised.com> wrote:

That is good news. I have been lening that way since the other versions are
either not 'enterprise' or have license issues. We'll look into the 10.04
release for Ubuntu Server - I assume Alestic will be making an AMI package
for it, also?

Doug.

Mark Korver
CTO/SpatialCloud.com