Hi List
(attempt 3 at sending this message, apologies if all three come through!)
I recently upgraded a Geonetwork 2.10.4 installation to the latest snapshot
to apply some security patches. I'm using tomcat6 and my approach was to
create a new database (postgresql) and restore a snapshot of my live one to
that. I copied my live data directory to a new one. I deployed a renamed
war file and set the config to point at the new database and new data
directory. This all worked fine, so I stopped tomcat6, and renamed
everything so that the new version was the one in use. The configuration
all seemed fine for a couple of days but then I started to get lots of
problems with both the spatial index and the lucene index (separate message
to follow about lucene).
For the spatial index- previously I was using postgresql for my database
but a shapefile spatial index. No problems were reported in the old
installation, but fairly quickly after doing the upgrade I started to see
errors about corruptions in the index. There were lots looking like the one
below but for different metadata IDs.
2015-07-20 02:06:24,337 ERROR [geonetwork.index] - Failure resetting
spatial index.
java.io.IOException: Dbf has extra record
at
org.geotools.data.shapefile.ShapefileAttributeReader.internalReadersHaveNext(ShapefileAttributeReader.java:153)
at
org.geotools.data.shapefile.ShapefileFeatureWriter.close(ShapefileFeatureWriter.java:237)
at
org.geotools.data.shapefile.indexed.IndexedShapefileFeatureWriter.close(IndexedShapefileFeatureWriter.java:109)
at
org.geotools.data.InProcessLockingManager$1.close(InProcessLockingManager.java:350)
at
org.geotools.data.AbstractFeatureStore.removeFeatures(AbstractFeatureStore.java:370)
at
org.fao.geonet.kernel.search.spatial.SpatialIndexWriter.reset(SpatialIndexWriter.java:276)
at
org.fao.geonet.kernel.search.SearchManager.setupIndex(SearchManager.java:1348)
at
org.fao.geonet.kernel.search.SearchManager.initLucene(SearchManager.java:648)
at org.fao.geonet.kernel.search.SearchManager.<init>(SearchManager.java:542)
at org.fao.geonet.Geonetwork.start(Geonetwork.java:320)
at jeeves.server.JeevesEngine.initAppHandler(JeevesEngine.java:605)
at jeeves.server.JeevesEngine.init(JeevesEngine.java:206)
at jeeves.server.sources.http.JeevesServlet.init(JeevesServlet.java:91)
at javax.servlet.GenericServlet.init(GenericServlet.java:212)
at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1213)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1026)
at
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4425)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4738)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943)
at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at org.apache.catalina.core.StandardService.start(StandardService.java:525)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
2015-07-20 02:06:24,429 INFO [jeeves.apphand] - - Access manager...
<snip>
2015-07-20 02:06:35,114 ERROR [geonetwork.index] - Failed to properly index
geometry of metadata 5422. Error: Dbf has extra record
java.io.IOException: Dbf has extra record
at
org.geotools.data.shapefile.ShapefileAttributeReader.internalReadersHaveNext(ShapefileAttributeReader.java:153)
at
org.geotools.data.shapefile.ShapefileFeatureWriter.close(ShapefileFeatureWriter.java:237)
at
org.geotools.data.shapefile.indexed.IndexedShapefileFeatureWriter.close(IndexedShapefileFeatureWriter.java:109)
at
org.geotools.data.InProcessLockingManager$1.close(InProcessLockingManager.java:350)
at
org.geotools.data.FilteringFeatureWriter.close(FilteringFeatureWriter.java:151)
at
org.geotools.data.AbstractFeatureStore.removeFeatures(AbstractFeatureStore.java:370)
at
org.fao.geonet.kernel.search.spatial.SpatialIndexWriter.delete(SpatialIndexWriter.java:221)
at
org.fao.geonet.kernel.search.SearchManager.indexGeometry(SearchManager.java:711)
at org.fao.geonet.kernel.search.SearchManager.index(SearchManager.java:695)
at org.fao.geonet.kernel.DataManager.indexMetadata(DataManager.java:580)
at
org.fao.geonet.kernel.DataManager$IndexMetadataTask.run(DataManager.java:399)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-07-20 02:06:35,115 ERROR [geonetwork.datamanager] - The metadata
document index with id=5422 is corrupt/invalid - ignoring it. Error: null
2015-07-20 02:06:35,158 ERROR [geonetwork.index] - Failed to properly index
geometry of metadata 5909. Error: Dbf has extra record
java.io.IOException: Dbf has extra record
at
org.geotools.data.shapefile.ShapefileAttributeReader.internalReadersHaveNext(ShapefileAttributeReader.java:153)
at
org.geotools.data.shapefile.ShapefileFeatureWriter.close(ShapefileFeatureWriter.java:237)
at
org.geotools.data.shapefile.indexed.IndexedShapefileFeatureWriter.close(IndexedShapefileFeatureWriter.java:109)
at
org.geotools.data.InProcessLockingManager$1.close(InProcessLockingManager.java:350)
at
org.geotools.data.FilteringFeatureWriter.close(FilteringFeatureWriter.java:151)
at
org.geotools.data.AbstractFeatureStore.removeFeatures(AbstractFeatureStore.java:370)
at
org.fao.geonet.kernel.search.spatial.SpatialIndexWriter.delete(SpatialIndexWriter.java:221)
at
org.fao.geonet.kernel.search.SearchManager.indexGeometry(SearchManager.java:711)
at org.fao.geonet.kernel.search.SearchManager.index(SearchManager.java:695)
at org.fao.geonet.kernel.DataManager.indexMetadata(DataManager.java:580)
at
org.fao.geonet.kernel.DataManager$IndexMetadataTask.run(DataManager.java:399)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-07-20 02:06:35,158 ERROR [geonetwork.datamanager] - The metadata
document index with id=5909 is corrupt/invalid - ignoring it. Error: null
As I couldn't see any documentation on how I get around this, I thought
that if I converted to using a postgis index it would rebuild everything
and I would stop getting the errors. I altered the connection string in
config.xml to jdbc:postgresql_postGIS://host:5432/database. The
documentation suggests this should be enough to get Geonetwork to use
postgis for the spatial index. I created the spatialindex table using the
setup scripts from the installation and when I restarted Geonetwork this
table was populated. However, when my server has restarted since then, it
has reverted to using shapefiles for it's index and I'm seeing the
corruption again. I can't see any errors about why it would revert back to
using shapefiles.
Can anyone give me some clues as to what's going on?
Thanks
Jo
--
*Jo Cook*
Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18
7RL, UK
t:+44 7930 524 155
iShare - Data integration and publishing platform
<http://www.isharemaps.com/>
*****************************************
Company registration no. 5410695. Registered in England and Wales.
Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
864201149.