Working with large Cloud Optimized GeoTiFF (COG) dataset in Geoserver

nikkopante · June 24, 2025, 8:23am

Hi I am trying to add Cloud Optimized GeoTiFF (COG) into my Geoserver instance. I used the geoserver-docker build using 2.27.x version. My COG is uploaded in Google Cloud Storage (GCS). I have tried adding smaller dataset and found no issue. However, when I am trying to add larger data <100 MB, the server does NOT STOP loading, seen from below image.

There are no logs found for this issue.

Does anyone know what is this issue about and how can I resolve it?

When I shut the container, and restart it. I will find the data added. However, when I add it to Layers, it will again not stop loading. Thus, I am not able to save add it as as layer.

When I try to delete the store, there will be an error.

java.lang.RuntimeException: Failed to upgrade lock from read to write state, please re-try the configuration operation
	at org.geoserver.GeoServerConfigurationLock.tryUpgradeLock(GeoServerConfigurationLock.java:192)
	at org.geoserver.catalog.LockingCatalogFacade.invoke(LockingCatalogFacade.java:34)
	at jdk.proxy3/jdk.proxy3.$Proxy41.remove(Unknown Source)
	at org.geoserver.catalog.impl.CatalogImpl.remove(CatalogImpl.java:226)
	at org.geoserver.security.SecureCatalogImpl.remove(SecureCatalogImpl.java:1367)
	at org.geoserver.catalog.impl.AbstractFilteredCatalog.remove(AbstractFilteredCatalog.java:770)
	at org.geoserver.catalog.impl.AbstractCatalogDecorator.remove(AbstractCatalogDecorator.java:97)
	at org.geoserver.catalog.CascadeDeleteVisitor.visitStore(CascadeDeleteVisitor.java:89)
	at org.geoserver.catalog.CascadeDeleteVisitor.visit(CascadeDeleteVisitor.java:99)
	at org.geoserver.catalog.impl.ModificationProxy.accept(ModificationProxy.java:187)
	at org.geoserver.catalog.impl.ModificationProxy.invoke(ModificationProxy.java:147)
	at jdk.proxy3/jdk.proxy3.$Proxy46.accept(Unknown Source)
	at org.geoserver.web.data.SelectionRemovalLink$1.onSubmit(SelectionRemovalLink.java:66)
	at org.geoserver.web.wicket.GeoServerDialog.submit(GeoServerDialog.java:99)
	at org.geoserver.web.wicket.GeoServerDialog$1.onSubmit(GeoServerDialog.java:110)
	at org.apache.wicket.ajax.markup.html.form.AjaxSubmitLink$1.onSubmit(AjaxSubmitLink.java:112)
	at org.apache.wicket.ajax.form.AjaxFormSubmitBehavior$AjaxFormSubmitter.onSubmit(AjaxFormSubmitBehavior.java:249)
	at org.apache.wicket.markup.html.form.Form.delegateSubmit(Form.java:1374)
	at org.apache.wicket.markup.html.form.Form.process(Form.java:1036)
	at org.apache.wicket.markup.html.form.Form.onFormSubmitted(Form.java:860)
	at org.apache.wicket.ajax.form.AjaxFormSubmitBehavior.onEvent(AjaxFormSubmitBehavior.java:202)
	at org.apache.wicket.ajax.AjaxEventBehavior.respond(AjaxEventBehavior.java:154)
	at org.apache.wicket.ajax.AbstractDefaultAjaxBehavior.onRequest(AbstractDefaultAjaxBehavior.java:630)
	at org.apache.wicket.core.request.handler.ListenerRequestHandler.internalInvoke(ListenerRequestHandler.java:300)
	at org.apache.wicket.core.request.handler.ListenerRequestHandler.invoke(ListenerRequestHandler.java:274)
	at org.apache.wicket.core.request.handler.ListenerRequestHandler.invokeListener(ListenerRequestHandler.java:222)
	at org.apache.wicket.core.request.handler.ListenerRequestHandler.respond(ListenerRequestHandler.java:202)
	at org.apache.wicket.request.cycle.RequestCycle$HandlerExecutor.respond(RequestCycle.java:910)
	at org.apache.wicket.request.RequestHandlerExecutor.execute(RequestHandlerExecutor.java:63)
	at org.apache.wicket.request.cycle.RequestCycle.execute(RequestCycle.java:294)
	at org.apache.wicket.request.cycle.RequestCycle.processRequest(RequestCycle.java:255)
	at org.apache.wicket.protocol.http.WicketFilter.processRequestCycle(WicketFilter.java:277)
	at org.apache.wicket.protocol.http.WicketFilter.processRequest(WicketFilter.java:208)
	at org.apache.wicket.protocol.http.WicketServlet.doPost(WicketServlet.java:159)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:555)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:623)
	at org.springframework.web.servlet.mvc.ServletWrappingController.handleRequestInternal(ServletWrappingController.java:166)
	at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:177)
	at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:51)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1072)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:965)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:555)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:623)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:199)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.apache.catalina.filters.CorsFilter.handleSimpleCORS(CorsFilter.java:240)
	at org.apache.catalina.filters.CorsFilter.doFilter(CorsFilter.java:152)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.geoserver.filters.ThreadLocalsCleanupFilter.doFilter(ThreadLocalsCleanupFilter.java:28)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:70)
	at org.geoserver.ows.HTTPHeadersCollector.doFilter(HTTPHeadersCollector.java:48)
	at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:67)
	at org.geoserver.filters.HTTPMethodFilter.doFilter(HTTPMethodFilter.java:36)
	at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:67)
	at org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:181)
	at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:67)
	at org.geoserver.filters.SpringDelegatingFilter.doFilter(SpringDelegatingFilter.java:41)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.geoserver.platform.AdvancedDispatchFilter.doFilter(AdvancedDispatchFilter.java:39)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:352)
	at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
	at org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:100)
	at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
	at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:89)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120)
	at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
	at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:89)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.geoserver.security.filter.GeoServerAnonymousAuthenticationFilter.doFilter(GeoServerAnonymousAuthenticationFilter.java:52)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:227)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:221)
	at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
	at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:89)
	at org.geoserver.security.filter.GeoServerUserNamePasswordAuthenticationFilter.doFilter(GeoServerUserNamePasswordAuthenticationFilter.java:117)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
	at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:110)
	at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:101)
	at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
	at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:89)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
	at org.geoserver.security.filter.GeoServerSecurityContextPersistenceFilter$1.doFilterInternal(GeoServerSecurityContextPersistenceFilter.java:66)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
	at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:89)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:225)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:190)
	at org.geoserver.security.GeoServerSecurityFilterChainProxy.doFilter(GeoServerSecurityFilterChainProxy.java:139)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.geoserver.filters.SecurityHeadersFilter.doFilter(SecurityHeadersFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:47)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.geoserver.filters.SessionDebugFilter.doFilter(SessionDebugFilter.java:48)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.geoserver.filters.FlushSafeFilter.doFilter(FlushSafeFilter.java:42)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:656)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:346)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:397)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:935)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1792)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1189)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:658)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
	at java.base/java.lang.Thread.run(Thread.java:840)

aaime-geosolutions · June 24, 2025, 9:43am

Known issue with large COG, the current code is slow to read the header, it’s reading small chunks and seems to never end.

Try adding the following system variable to the JVM running GeoServer (forces it to read in chunks of 256Kb):
-Dit.geosolutions.cog.default.header.length=262144

For anyone reading, we’re looking for funding (or developer coding time) to improve this so that the code would automatically increase the read size and thus work fine out of the box. If anyone is interested, please let me know, it’s one of the two issues keeping COG in unsupported land.

Cheers
Andrea

nikkopante · June 26, 2025, 3:45am

Thank you @aaime-geosolutions!

I have tried adding the additional JVM configuration. However, the problem persisted after adding another large COG dataset.

I do not have much financially. I am interested to contribute as a developer. Please let me know how can I help.

Thank you!

aaime-geosolutions · June 26, 2025, 7:38am

I can only think of two causes, the JVM option did not work or the file is not actually a COG

The low level COG reading machinery is here in imageio-ext:

And specifically for GS (Google Storage) the reader is here:

github.com

geosolutions-it/imageio-ext/blob/master/plugin/cog/cog-rangereader-gs/src/main/java/it/geosolutions/imageioimpl/plugins/cog/GSRangeReader.java#L123


      
                  String key = path.substring(idx + 1);
                  id = BlobId.of(bucket, key);
              } else {
                  throw new IllegalArgumentException("Don't know how to process GS link: " + uri);
              }
              return id;
          }
          
          @Override
          public byte[] fetchHeader() {
              byte[] currentHeader = data.get(0L);
              if (currentHeader != null) {
                  headerOffset = currentHeader.length;
              }
          
              byte[] headerBytes = readInternal(headerOffset, headerLength);
              data.put(0L, headerBytes);
              return headerBytes;
          }
          
          byte[] readInternal(long readOffset, int readLength) {

In case of large headers (of files that are not really COG, and don’t have the header at the beginning of the stream) fetchHeader is called repeatedly.
The machinery should be modified to either recognize the header length during the process, or increase the reading length in a controlled way so that one does not need to perform many small reads to fetch a large header.
We don’t have a development plan right now (that would happen as we secure funding, if you go for developing directly, make some research and propose an approach).

Thank you!
[/quote]