[Geoserver-devel] Using soft caches in resource pool

Andrea_Aime4 · August 22, 2011, 9:17am

Hi,
the current ResourcePool implementation uses commons collection LRUMap
for caching
expensive to create resources such as datastores (think connection
pool) or coverage readers.
The LRU map has a max size, normally equal to 100, when that is exceeded the old
entries get removed and disposed of.
This has two issues:
* under heavy load a resource can be disposed while still in use from
a request thread
  (it actually requires a very long request or a heavy number of
requests all accessing
   different resources, but it's still possible)
* if a server has plenty of resources the limit will still be 100,
regardless of the actual
  available memory

The patch attached to GEOS-4730 switches from LRUMap from SoftValueHashMap,
still properly handling the disposal of resources, but without the two
above limitations:
if there is plenty of memory the resources exceeding the 100 count
will be stored as
soft references that might last a while, and a resource in use will never become
softly reachable, so there is no risk of having that one being disposed while a
request is in flight.

I'm proposing to apply the patch for a while on trunk, and eventually
decide whether
to backport it to the stable series later.

Mind, there is still another issue, if a reset is called upon by the admin
the resources will be forcefully disposed of anyways, but I guess that
for that one we'll
need a entirely different way to pool them (especially if we consider
also async request
which, with wps, sooner or later will come onto our plate).
That's another story, which we'll handle when there are enough
resources on the plate.

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

Justin_Deoliveira · August 22, 2011, 3:02pm

I think the patch sounds good, but I thought the issue with using a soft reference based cache was that we could never be ensured that we got the dispose method called. Or at least we could not be sure that he object was still around. Scanned thought the machinery of soft hash map and still not 100% sure.

On Mon, Aug 22, 2011 at 3:17 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
the current ResourcePool implementation uses commons collection LRUMap
for caching
expensive to create resources such as datastores (think connection
pool) or coverage readers.
The LRU map has a max size, normally equal to 100, when that is exceeded the old
entries get removed and disposed of.
This has two issues:

under heavy load a resource can be disposed while still in use from
a request thread
(it actually requires a very long request or a heavy number of
requests all accessing
different resources, but it’s still possible)

if a server has plenty of resources the limit will still be 100,
regardless of the actual
available memory

The patch attached to GEOS-4730 switches from LRUMap from SoftValueHashMap,
still properly handling the disposal of resources, but without the two
above limitations:
if there is plenty of memory the resources exceeding the 100 count
will be stored as
soft references that might last a while, and a resource in use will never become
softly reachable, so there is no risk of having that one being disposed while a
request is in flight.

I’m proposing to apply the patch for a while on trunk, and eventually
decide whether
to backport it to the stable series later.

Mind, there is still another issue, if a reset is called upon by the admin
the resources will be forcefully disposed of anyways, but I guess that
for that one we’ll
need a entirely different way to pool them (especially if we consider
also async request
which, with wps, sooner or later will come onto our plate).
That’s another story, which we’ll handle when there are enough
resources on the plate.

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

uberSVN’s rich system and user administration capabilities and model
configuration take the hassle out of deploying and managing Subversion and
the tools developers use with it. Learn more about uberSVN and get a free
download at: http://p.sf.net/sfu/wandisco-dev2dev

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Andrea_Aime4 · August 22, 2011, 4:31pm

On Mon, Aug 22, 2011 at 5:02 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

I think the patch sounds good, but I thought the issue with using a soft
reference based cache was that we could never be ensured that we got the
dispose method called. Or at least we could not be sure that he object was
still around. Scanned thought the machinery of soft hash map and still not
100% sure.

The SoftHashMap has a "cleaner" that you can set to dispose of the referenced
objects while the soft reference is getting phased out by the garbage collector.
It is already in place but would not always work so I also made a
couple of modifications
in GeoTools to make it work always (they are in a correspondent geotools patch
that I failed to publish).

If you look at the patch the base class of all the resource caches implements
a custom cleaner which then delegates to an abstract method that subclasses
implement.

The hard part is that testing for a soft reference to be removed is hard,
the moment the reference becomes soft there is no way to tell how many
times System.gc() needs to be called in order for that reference to actually
be removed...
Hmm... maybe I can use this and try to force (quickly) an OOM to prove
soft reference disposal actually works:
http://stackoverflow.com/questions/457229/how-to-cause-soft-references-to-be-cleared-in-java

This is going to be nasty if we start using parallel builds, that OOM is
going to create havoc in tests run in parallel...