ConcurrentModificationException in DefaultCatalogFacade.getLayerByName
----------------------------------------------------------------------
Key: GEOS-4404
URL: http://jira.codehaus.org/browse/GEOS-4404
Project: GeoServer
Issue Type: Bug
Components: Global
Affects Versions: 2.1-RC2, 2.1-RC1
Reporter: Eli Miller
Assignee: Andrea Aime
Attachments: fix1.patch, test1.patch
We have encountered situations where GeoServer requests fail due to ConcurrentModificationException. The problem is intermittent and only happens when there are administrative requests being made. The underlying problem is related to DefaultCatalogFacade not handling concurrency on the layers field. Two obvious solutions to this problem would be to synchronize layers while iterating (assuming that all other mutators are also synchronizing) or to use a concurrent data structure. Given the read-biased nature of the data, it seems like the most efficient solution would be to use a concurrent data structure. The following stacktraces illustrate the problem that we have been seeing.
java.util.ConcurrentModificationException
at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
at java.util.AbstractList$Itr.next(AbstractList.java:343)
at org.geoserver.catalog.impl.DefaultCatalogFacade.getLayerByName(DefaultCatalogFacade.java:422)
at org.geoserver.catalog.impl.CatalogImpl.getLayerByName(CatalogImpl.java:701)
at org.geoserver.security.SecureCatalogImpl.getLayerByName(SecureCatalogImpl.java:262)
at org.geoserver.wms.WMS.getLayerByName(WMS.java:141)
at org.geoserver.wms.map.GetMapKvpRequestReader.parseLayers(GetMapKvpRequestReader.java:1155)
at org.geoserver.wms.map.GetMapKvpRequestReader.read(GetMapKvpRequestReader.java:204)
at org.geoserver.wms.map.GetMapKvpRequestReader.read(GetMapKvpRequestReader.java:74)
at org.geoserver.ows.Dispatcher.parseRequestKVP(Dispatcher.java:1144)
at org.geoserver.ows.Dispatcher.dispatch(Dispatcher.java:482)
at org.geoserver.ows.Dispatcher.handleRequestInternal(Dispatcher.java:226)
Caused by: java.util.ConcurrentModificationException
at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
at java.util.AbstractList$Itr.next(AbstractList.java:343)
at org.geoserver.catalog.impl.DefaultCatalogFacade.getLayerByName(DefaultCatalogFacade.java:422)
at org.geoserver.catalog.impl.CatalogImpl.getLayerByName(CatalogImpl.java:701)
at org.geoserver.catalog.impl.CatalogImpl.validate(CatalogImpl.java:621)
at org.geoserver.catalog.impl.CatalogImpl.add(CatalogImpl.java:596)
at org.geoserver.security.SecureCatalogImpl.add(SecureCatalogImpl.java:864)
at org.geoserver.catalog.rest.CoverageResource.handleObjectPost(CoverageResource.java:96)
at org.geoserver.rest.ReflectiveResource.handlePost(ReflectiveResource.java:122)
... 222 more
I have attached a patch for CatalogImplTest that illustrates some of these concurrency issues. Unfortunately, the test will not always fail because it depends on threading behavior particular to individual systems and load. I can generally get the test to fail on every fourth run but you may need to adjust the values of GET_LAYER_BY_ID_WITH_CONCURRENT_ADD_TEST_COUNT and/or GET_LAYER_BY_ID_WITH_CONCURRENT_ADD_THREAD_COUNT to coax out a failure on your system.
I have also attached a patch for DefaultCatalogFacade that changes layers from ArrayList to CopyOnWriteArrayList and removes synchronizations that are made obsolete by this change. After making these changes I have not been able to generate a test failure.
FWIW, based on a cursory review of DefaultCatalogFacade there seem to be other data constructs with similar problems. I will try to put together some more tests and fixes for these scenarios.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira