Hi,
today I've tried to give another shot at the redeploy issue.
What issue? Well, Geoserver uses a lot of pergen memory to store
all of its classes bytecode, and when the application gets undeployed,
this bytecode does not get unloaded.
Result: 2/3 redeploys and your web app container needs to be restarted,
as it starts throwing OOM in the permgen space
Why is this happening? Well, because of a tangle of references.
Each class of GeoServer is loaded by a custom web application class loader provided by the container (so that all web apps are isolated
from each other).
Each class loaded points back to the class loader.
When an application is unloaded the class loader is thrown away, but
it may not be garbage collected if there is still something referencing
it. That is, the classes it loaded... which are not garbage collected
if there is something outside of the web class loader pointing at them?
What? Well, a ton of things:
- JDK global registries (Image I/O, JAI, JDBC driver manager)
- threads created by the web app
- thread locals that are not removed and are still attached to
the thread pool that Tomcat is using to serve the requests
So, in theory, when the app goes down we should perform some cleanup.
I tried my best to do that using a servlet listener contextDestroyed
method, cleaning everything that I knew was related (patch attached,
not too elegant, would like to move part of that code to a GeoTools
shutdown method, started a mail thread about that on gt-devel)
I also used Tomcat 6.0.26 and its JRE Memory Leak Prevention listener
(http://tomcat.apache.org/tomcat-6.0-doc/config/listeners.html) which
reports leak sources... still, no joy, the leak is still there.
I'm pretty sure there are no surviving threads.
And that the thread locals are cleaned up by Tomcat.
Still... no luck, there is still something registered somewhere
that's keeping a finger on the GeoServer bytecode...
any idea what that might be?
For example, is there any XML related registry that we register
into and that should be cleaned up?
Anything else?
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
(attachments)
cleanup.patch (14.1 KB)