[Geoserver-devel] [JIRA] (GEOS-10496) Using the REST API to purge NetCDF granules causes a seemingly infinite loop

Andrea Aime created an issue

GeoServer / TaskGEOS-10496

Using the REST API to purge NetCDF granules causes a seemingly infinite loop

Issue Type:

TaskTask

Assignee:

Andrea Aime

Created:

19/May/22 12:21 PM

Priority:

MediumMedium

Reporter:

Andrea Aime

Trying to delete all the granules related to a given NetCDF files, like this:

curl -XDELETE -u admin:geoserver “http://localhost:8080/geoserver/rest/workspaces/rm/coveragestores/hsf_ita/coverages/mycoverage/index/granules?filter=$filter&purge=metadata

Seems to end up with an infinite loop of sorts once the last coverage of the NetCDF file is being purged (assuming there are multiple ones), with the NetCDF file re-populating its index as the granule is processed for deletion:

at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:321)
        - locked <0x00000007656de580> (a org.postgresql.core.v3.QueryExecutorImpl)
        at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:473)
        at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:393)
        at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:322)
        at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:308)
        at org.postgresql.jdbc.PgStatement.executeUpdate(PgStatement.java:1233)
        at org.postgresql.jdbc.PgStatement.executeUpdate(PgStatement.java:1215)
        at org.apache.commons.dbcp.DelegatingStatement.executeUpdate(DelegatingStatement.java:323)
        at org.apache.commons.dbcp.DelegatingStatement.executeUpdate(DelegatingStatement.java:323)
        at org.geotools.jdbc.JDBCDataStore.insertNonPS(JDBCDataStore.java:1909)
        at org.geotools.jdbc.JDBCDataStore.insert(JDBCDataStore.java:1731)
        - locked <0x0000000765613ac8> (a org.geotools.jdbc.JDBCDataStore)
        at org.geotools.jdbc.JDBCInsertFeatureWriter.flush(JDBCInsertFeatureWriter.java:126)
        at org.geotools.jdbc.JDBCInsertFeatureWriter.write(JDBCInsertFeatureWriter.java:98)
        at org.geotools.data.InProcessLockingManager$1.write(InProcessLockingManager.java:296)
        at org.geotools.data.store.ContentFeatureStore.addFeature(ContentFeatureStore.java:302)
        at org.geotools.data.store.ContentFeatureStore.addFeatures(ContentFeatureStore.java:254)
        at org.geotools.coverage.io.catalog.CoverageSlicesCatalog.addGranules(CoverageSlicesCatalog.java:384)
        at org.geotools.imageio.netcdf.NetCDFImageReader.initIndex(NetCDFImageReader.java:388)
        at org.geotools.imageio.netcdf.NetCDFImageReader.init(NetCDFImageReader.java:538)
        at org.geotools.imageio.netcdf.NetCDFImageReader.setInput(NetCDFImageReader.java:303)
        at javax.imageio.ImageReader.setInput(ImageReader.java:380)
        at org.geotools.coverage.io.netcdf.NetCDFAccess.<init>(NetCDFAccess.java:119)
        at org.geotools.coverage.io.netcdf.NetCDFDriver.connect(NetCDFDriver.java:92)
        at org.geotools.coverage.io.impl.DefaultFileDriver.process(DefaultFileDriver.java:228)
        at org.geotools.coverage.io.netcdf.NetCDFReader.<init>(NetCDFReader.java:158)
        at org.geotools.coverage.io.netcdf.NetCDFFormat.getReader(NetCDFFormat.java:94)
        at org.geotools.gce.imagemosaic.GranuleDescriptor.init(GranuleDescriptor.java:395)
        at org.geotools.gce.imagemosaic.GranuleDescriptor.<init>(GranuleDescriptor.java:915)
        at org.geotools.gce.imagemosaic.catalog.CachingDataStoreGranuleCatalog.getGranuleDescriptor(CachingDataStoreGranuleCatalog.java:207)
        at org.geotools.gce.imagemosaic.catalog.CachingDataStoreGranuleCatalog.getGranuleDescriptors(CachingDataStoreGranuleCatalog.java:172)
        at org.geotools.gce.imagemosaic.PurgingGranuleStore.removeGranules(PurgingGranuleStore.java:104)
        at org.geoserver.rest.catalog.StructuredCoverageController.granulesDeleteInternal(StructuredCoverageController.java:249)
        at org.geoserver.rest.catalog.StructuredCoverageController.granulesDelete(StructuredCoverageController.java:159)

The purging code goes through the granule descriptor which in turn initializes the store, which I guess finds the database empty and re-initializes itself… we probably need to either avoid going through the granule descriptor (just to find the raster URL really) or have hints that avoid re-initialization of the NetCDF.

Also noticing, the code is assuming granule==file but for NetCDF one file maps to many granules, it should probably be processed once only.

Add Comment

Add Comment

Get Jira notifications on your phone! Download the Jira Cloud app for Android or iOS


This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100198-sha1:0f60739)

Atlassian logo