Hi List
(attempt 3 at sending this message, apologies if all three come through!)
I recently upgraded a Geonetwork 2.10.4 installation to the latest snapshot
to apply some security patches. I'm using tomcat6 and my approach was to
create a new database (postgresql) and restore a snapshot of my live one to
that. I copied my live data directory to a new one. I deployed a renamed
war file and set the config to point at the new database and new data
directory. This all worked fine, so I stopped tomcat6, and renamed
everything so that the new version was the one in use. The configuration
all seemed fine for a couple of days but then I started to get lots of
problems with both the spatial index and the lucene index (separate message
has been sent about the spatial index).
These are the log entries relating to lucene at startup:
2015-07-20 02:05:46,719 INFO [jeeves.apphand] - Lucene configuration:
* Version: LUCENE_36
* RAMBufferSize: 48.0
* MergeFactor: 10
* Default analyzer: org.apache.lucene.analysis.WhitespaceAnalyzer
* Field analyzers:
{_source=org.apache.lucene.analysis.core.KeywordAnalyzer,
hassource=org.fao.geonet.kernel.search.GeoNetworkAnalyzer,
parentUuid=org.fao.geonet.kernel.search.GeoNetworkAnalyzer,
_groupPublished=org.apache.lucene.analysis.core.KeywordAnalyzer,
subject=org.apache.lucene.analysis.core.KeywordAnalyzer,
hasfeaturecat=org.fao.geonet.kernel.search.GeoNetworkAnalyzer,
keyword=org.apache.lucene.analysis.core.KeywordAnalyzer,
any=org.fao.geonet.kernel.search.GeoNetworkAnalyzer,
_cat=org.apache.lucene.analysis.core.KeywordAnalyzer,
operatesOnIdentifier=org.fao.geonet.kernel.search.GeoNetworkAnalyzer,
operatesOn=org.fao.geonet.kernel.search.GeoNetworkAnalyzer,
_uuid=org.fao.geonet.kernel.search.GeoNetworkAnalyzer,
inspirerelated=org.apache.lucene.analysis.core.KeywordAnalyzer}
* Field search analyzers:
{_source=org.apache.lucene.analysis.core.KeywordAnalyzer,
parentUuid=org.fao.geonet.kernel.search.GeoNetworkAnalyzer,
hassource=org.fao.geonet.kernel.search.GeoNetworkAnalyzer,
subject=org.apache.lucene.analysis.core.KeywordAnalyzer,
_groupPublished=org.apache.lucene.analysis.core.KeywordAnalyzer,
keyword=org.apache.lucene.analysis.core.KeywordAnalyzer,
operatesOn=org.fao.geonet.kernel.search.GeoNetworkAnalyzer,
hasfeaturecat=org.fao.geonet.kernel.search.GeoNetworkAnalyzer,
any=org.apache.lucene.analysis.WhitespaceAnalyzer,
_cat=org.apache.lucene.analysis.core.KeywordAnalyzer,
operatesOnIdentifier=org.fao.geonet.kernel.search.GeoNetworkAnalyzer,
_uuid=org.fao.geonet.kernel.search.GeoNetworkAnalyzer,
inspirerelated=org.apache.lucene.analysis.core.KeywordAnalyzer}
* Field boost factor: {_op2=0.0, _owner=0.0, _isTemplate=0.0, _op0=0.0,
_op1=0.0, _dummy=0.0}
* Boost document class: null
* Tokenized fields: [abstract, keywordType, title, subject, any,
levelName, altTitle, specificationTitle]
* Numeric fields: [eastBL, southBL, northBL, westBL, denominator]
* Dump fields:
{_valid_schematron-rules-inspire=valid_schematron-rules-inspire,
_popularity=popularity, _dynamic=dynamic, _source=source, _owner=owner,
accessConstr=Constraints, link=link, _ownername=ownername, _view=view,
responsibleParty=responsibleParty, type=type, _userinfo=userinfo,
_locale=_locale, _valid_schematron-rules-iso=valid_schematron-rules-iso,
_selected=selected, _isTemplate=isTemplate,
_isPublishedToAll=isPublishedToAll, _valid=valid,
classif=SecurityConstraints, _rating=rating, _cat=category, credit=credit,
otherConstr=Constraints, _status=status, _title=title, abstract=abstract,
_featured=featured,
_valid_schematron-rules-geonetwork=valid_schematron-rules-geonetwork,
parentUuid=parentId, keyword=keyword, image=image, datasetLang=datasetLang,
conditionApplyingToAccessAndUse=Constraints, _displayOrder=displayOrder,
_indexingErrorMsg=idxMsg, serviceType=serviceType, geoBox=geoBox,
_isHarvested=isHarvested,
spatialRepresentationType=spatialRepresentationType, _download=download,
_valid_xsd=valid_xsd, _indexingError=idxError, language=language,
_defaultTitle=defaultTitle, _edit=edit, _docLocale=docLocale,
_notify=notify}
* Search boost query: null
* Score:
* trackDocScores: false
* trackMaxScore: false
* docsScoredInOrder: false
Taxonomy configuration: [hits, results, titles, results_with_summary]
* type: hits
{inspiretheme=Field: inspiretheme name:inspireTheme max:35 sort byVALUE sort
order:ASCENDIND, metadataPOC=Field: metadataPOC name:metadataPOC max:15 sort
byCOUNT sort order:DESCENDING, spatialRepresentationType=Field:
spatialRepresentationType name:spatialRepresentationType max:10 sort
byCOUNT sort
order:DESCENDING, keyword=Field: keyword name:keyword max:15 sort byCOUNT sort
order:DESCENDING, createDateYear=Field: createDateYear name:createDateYear
max:40 sort byVALUE sort order:DESCENDING, denominator=Field: denominator
name:denominator max:10 sort byNUMVALUE sort order:DESCENDING, type=Field:
type name:type max:10 sort byCOUNT sort order:DESCENDING,
serviceType=Field: serviceType name:serviceType max:10 sort byCOUNT sort
order:DESCENDING, orgName=Field: orgName name:orgName max:15 sort byCOUNT sort
order:DESCENDING} * type: results
{keyword=Field: keyword name:keyword max:100 sort byCOUNT sort
order:DESCENDING} * type: titles
{title=Field: title name:title max:100 sort byCOUNT sort order:DESCENDING}
* type: results_with_summary
{inspiretheme=Field: inspiretheme name:inspireTheme max:35 sort byVALUE sort
order:ASCENDIND, metadataPOC=Field: metadataPOC name:metadataPOC max:15 sort
byCOUNT sort order:DESCENDING, spatialRepresentationType=Field:
spatialRepresentationType name:spatialRepresentationType max:10 sort
byCOUNT sort
order:DESCENDING, keyword=Field: keyword name:keyword max:15 sort byCOUNT sort
order:DESCENDING, createDateYear=Field: createDateYear name:createDateYear
max:40 sort byCOUNT sort order:ASCENDIND, denominator=Field: denominator
name:denominator max:20 sort byVALUE sort order:ASCENDIND, type=Field: type
name:type max:10 sort byCOUNT sort order:DESCENDING, serviceType=Field:
serviceType name:serviceType max:10 sort byCOUNT sort order:DESCENDING,
orgName=Field: orgName name:orgName max:15 sort byCOUNT sort
order:DESCENDING}
2015-07-20 02:05:46,831 WARN [geonetwork.search] - Failed to create
analyzer as specified in lucene config, default analyzer will be used for
field null. Exception message is:
org.apache.lucene.analysis.WhitespaceAnalyzer
2015-07-20 02:05:46,832 WARN [geonetwork.search] - Creating analyzer has
failed, defaulting to GeoNetworkAnalyzer
2015-07-20 02:05:46,876 WARN [geonetwork.search] - Failed to create
analyzer as specified in lucene config, default analyzer will be used for
field null. Exception message is:
org.apache.lucene.analysis.WhitespaceAnalyzer
2015-07-20 02:05:46,877 WARN [geonetwork.search] - Creating analyzer has
failed, defaulting to GeoNetworkAnalyzer
(this continues for ~200 lines)
When people do a lucene search they sometimes get an error in the browser,
but also repeated searches for the same thing give different results (eg a
simple search might return 1 record, then you try it again (after resetting
the search) and it might return 0 results). I can see these sorts of errors
in the log file:
2015-07-20 02:05:49,479 ERROR [geonetwork.index] - An error occurred while
openning taxonomy readers/writers
org.apache.lucene.index.CorruptIndexException: failed to locate current
segments_N file
at
org.apache.lucene.index.IndexFileDeleter.<init>(IndexFileDeleter.java:224)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:704)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.openIndexWriter(DirectoryTaxonomyWriter.java:283)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.<init>(DirectoryTaxonomyWriter.java:210)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.<init>(DirectoryTaxonomyWriter.java:332)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.<init>(DirectoryTaxonomyWriter.java:348)
at
org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.init(TaxonomyIndexTracker.java:57)
at
org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.<init>(TaxonomyIndexTracker.java:44)
at
org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.<init>(LuceneIndexLanguageTracker.java:55)
at
org.fao.geonet.kernel.search.SearchManager.setupIndex(SearchManager.java:1318)
at
org.fao.geonet.kernel.search.SearchManager.initLucene(SearchManager.java:648)
at org.fao.geonet.kernel.search.SearchManager.<init>(SearchManager.java:542)
at org.fao.geonet.Geonetwork.start(Geonetwork.java:320)
at jeeves.server.JeevesEngine.initAppHandler(JeevesEngine.java:605)
at jeeves.server.JeevesEngine.init(JeevesEngine.java:206)
at jeeves.server.sources.http.JeevesServlet.init(JeevesServlet.java:91)
at javax.servlet.GenericServlet.init(GenericServlet.java:212)
at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1213)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1026)
at
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4425)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4738)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943)
at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at org.apache.catalina.core.StandardService.start(StandardService.java:525)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
2015-07-20 02:05:49,481 ERROR [geonetwork.lucene] - Failure while closing
luceneIndexLanguageTracker
java.lang.NullPointerException
at
org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.close(TaxonomyIndexTracker.java:129)
at
org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.init(TaxonomyIndexTracker.java:70)
at
org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.<init>(TaxonomyIndexTracker.java:44)
at
org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.<init>(LuceneIndexLanguageTracker.java:55)
at
org.fao.geonet.kernel.search.SearchManager.setupIndex(SearchManager.java:1318)
at
org.fao.geonet.kernel.search.SearchManager.initLucene(SearchManager.java:648)
at org.fao.geonet.kernel.search.SearchManager.<init>(SearchManager.java:542)
at org.fao.geonet.Geonetwork.start(Geonetwork.java:320)
at jeeves.server.JeevesEngine.initAppHandler(JeevesEngine.java:605)
at jeeves.server.JeevesEngine.init(JeevesEngine.java:206)
at jeeves.server.sources.http.JeevesServlet.init(JeevesServlet.java:91)
at javax.servlet.GenericServlet.init(GenericServlet.java:212)
at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1213)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1026)
at
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4425)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4738)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943)
at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at org.apache.catalina.core.StandardService.start(StandardService.java:525)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
2015-07-20 02:05:49,481 ERROR [geonetwork.index] - Exception while opening
lucene index, going to rebuild it:
org.apache.lucene.index.CorruptIndexException: failed to locate current
segments_N file
at
org.apache.lucene.index.IndexFileDeleter.<init>(IndexFileDeleter.java:224)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:704)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.openIndexWriter(DirectoryTaxonomyWriter.java:283)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.<init>(DirectoryTaxonomyWriter.java:210)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.<init>(DirectoryTaxonomyWriter.java:332)
at
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.<init>(DirectoryTaxonomyWriter.java:348)
at
org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.init(TaxonomyIndexTracker.java:57)
at
org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.<init>(TaxonomyIndexTracker.java:44)
at
org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.<init>(LuceneIndexLanguageTracker.java:55)
at
org.fao.geonet.kernel.search.SearchManager.setupIndex(SearchManager.java:1318)
at
org.fao.geonet.kernel.search.SearchManager.initLucene(SearchManager.java:648)
at org.fao.geonet.kernel.search.SearchManager.<init>(SearchManager.java:542)
at org.fao.geonet.Geonetwork.start(Geonetwork.java:320)
at jeeves.server.JeevesEngine.initAppHandler(JeevesEngine.java:605)
at jeeves.server.JeevesEngine.init(JeevesEngine.java:206)
at jeeves.server.sources.http.JeevesServlet.init(JeevesServlet.java:91)
at javax.servlet.GenericServlet.init(GenericServlet.java:212)
at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1213)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1026)
at
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4425)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4738)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943)
at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at org.apache.catalina.core.StandardService.start(StandardService.java:525)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
2015-07-20 02:05:49,482 ERROR [geonetwork.index] - Rebuilding lucene index
Rebuilding the index from the admin menu fixes the problem, but it returns-
first it happened about 4 days later but most recently it happened on
Friday and then again on Monday morning. No explicit errors have been
reported since Monday, but the odd search issue remains.
Any pointers would be gratefully received, otherwise I think I will need to
revert back to the old installation.
Regards
Jo
--
*Jo Cook*
Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18
7RL, UK
t:+44 7930 524 155
iShare - Data integration and publishing platform
<http://www.isharemaps.com/>
*****************************************
Company registration no. 5410695. Registered in England and Wales.
Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
864201149.