[Geoserver-devel] [JIRA] (GEOS-9674) Potential memory leak when shutting down Tomcat

Matthew Northcott created an issue

GeoServer / BugGEOS-9674

Potential memory leak when shutting down Tomcat

Issue Type:

BugBug

Affects Versions:

2.17.1

Assignee:

Unassigned

Created:

03/Jul/20 1:02 AM

Environment:

Ubuntu 18.04 64-bit
Docker version 19.03.12
OpenJDK 11
Tomcat 9

Priority:

MediumMedium

Reporter:

Matthew Northcott

Issue

There is a possible memory leak when shutting down GeoServer, indicated by Tomcat.

01-Jul-2020 14:27:07.488 SEVERE [Thread-19] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [geoserver] created a ThreadLocal with key of type [java.lang.ThreadLocal.SuppliedThreadLocal] (value [java.lang.ThreadLocal$SuppliedThreadLocal@anonymised.com]) and a value of type [org.geowebcache.storage.CompositeBlobStore.StoreSuitabilityCheck] (value [EXISTING]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
01-Jul-2020 14:27:07.490 SEVERE [Thread-19] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [geoserver] created a ThreadLocal with key of type [com.sun.jna.Structure$2] (value [com.sun.jna.Structure$2@anonymised.com]) and a value of type [com.sun.jna.Structure.StructureSet] (value [[]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
01-Jul-2020 14:27:07.491 SEVERE [Thread-19] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [geoserver] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@anonymised.com]) and a value of type [org.springframework.security.core.context.SecurityContextImpl] (value [org.springframework.security.core.context.SecurityContextImpl@anonymised.com: Null authentication]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

To reproduce, start GeoServer under Tomcat, let it start up and then shut down the server.

Findings

This may be a consequence of using thread pools and ThreadLocal variables (see https://stackoverflow.com/a/818120) whose impact is dependent on the number of redeployments (see https://stackoverflow.com/a/7791593).

Looking at these logs individually:

org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [geoserver] created a ThreadLocal with key of type [java.lang.ThreadLocal.SuppliedThreadLocal] (value [java.lang.ThreadLocal$SuppliedThreadLocal@anonymised.com]) and a value of type [org.geowebcache.storage.CompositeBlobStore.StoreSuitabilityCheck] (value [EXISTING]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

The referenced ThreadLocal is found at https://github.com/GeoWebCache/geowebcache/blob/master/geowebcache/core/src/main/java/org/geowebcache/storage/CompositeBlobStore.java#L101

org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [geoserver] created a ThreadLocal with key of type [com.sun.jna.Structure$2] (value [com.sun.jna.Structure$2@anonymised.com]) and a value of type [com.sun.jna.Structure.StructureSet] (value [[]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

The referenced ThreadLocal is found at https://github.com/java-native-access/jna/blob/65085c1a2fb85aa2f70823a2af1ac1e6b91288e1/src/com/sun/jna/Structure.java#L474

Wthin GeoServer, the only use of jna is as a dependency of OSHI. Specifically, the OSHISystemInfoCollector. From limited experimentation with this class, I found that commenting out the class attributes (of OSHI-defined types) resolves this specific message.

org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [geoserver] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@anonymised.com]) and a value of type [org.springframework.security.core.context.SecurityContextImpl] (value [org.springframework.security.core.context.SecurityContextImpl@anonymised.com7...: Null authentication]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

I did not research further into this one after reading a forum post on the topic (see http://osgeo-org.1560.x6.nabble.com/Memory-Leak-after-deploying-geoserver-war-tp5200785p5200794.html).

Related Issues

https://osgeo-org.atlassian.net/browse/GEOT-3330 - an old issue with describing a similar problem

Add Comment

Add Comment

Get Jira notifications on your phone! Download the Jira Cloud app for Android or iOS


This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100131-sha1:aac02e2)

Atlassian logo