[GeoNetwork-devel] Multinode Geonetwork

Hi Developers,

I’m forwarding this to the dev list to see if it gets a response- I’m really keen to get Multinode Geonetwork working and am happy to test/expand documentation if necessary.

Thanks

Jo

···

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Hi All,

Some further research on this- and two issues that might be related. The errors I’m seeing seem to be related to the Lucene Directory, which is currently specified in the tomcat options. My databases are both postgresql, and I’m seeing a lot of errors in the log files related to postgresql:

2016-10-10 14:56:18,131 DEBUG [org.springframework.orm.jpa.JpaTransactionManager] - Resuming suspended transaction after completion of inner transaction
2016-10-10 14:56:18,132 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Setting JDBC Connection [jdbc:postgresql://[myhost]:5432/[mydatabase], UserName=geonetwork3, PostgreSQL Native Driver] read-only
2016-10-10 14:56:18,132 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Could not set JDBC Connection read-only
org.postgresql.util.PSQLException: Cannot change transaction read-only property in the middle of a transaction.

Could these be related to the multinode errors I’m seeing?

Has anyone successfully set up a Geonetwork 3.2.x multinode server, with postgresql as the back end database that could give me some pointers?

Thanks

Jo

···

On Fri, Oct 7, 2016 at 3:03 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’m forwarding this to the dev list to see if it gets a response- I’m really keen to get Multinode Geonetwork working and am happy to test/expand documentation if necessary.

Thanks

Jo

---------- Forwarded message ----------
From: Jo Cook <jocook@anonymised.com>
Date: Thu, Oct 6, 2016 at 4:04 PM
Subject: Multinode Geonetwork
To: “geonetwork-users@anonymised.comsourceforge.net” <geonetwork-users@anonymised.comsourceforge.net>

Hi Developers,

I’m testing Geonetwork 3.2.x, specifically in multinode format. I’ve configured a second node using these instructions: http://geonetwork-opensource.org/manuals/trunk/eng/users/maintainer-guide/installing/using-multinode-mode.html

I’ve done this manually (ie adding a node to WEB-INF/web.xml and a node in config-node copied from the example xml in the link above), and also using the shell script.

The second database (postgres) is populated on a tomcat restart and I can visit the home page for the second node, which shows no metadata, as expected. When I sign in I get the following error in the browser:

Raised exception while starting the application. Fix the error and restart.

Message
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
Handler
org.fao.geonet.Geonetwork
Exception
java.lang.RuntimeException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
Stack
java.lang.RuntimeException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.lazyInit(LuceneIndexLanguageTracker.java:124)
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.acquire(LuceneIndexLanguageTracker.java:187)
at org.fao.geonet.kernel.search.SearchManager.getIndexReader(SearchManager.java:1210)
at org.fao.geonet.kernel.search.SearchManager.getNewIndexReader(SearchManager.java:1215)
at org.fao.geonet.kernel.search.SearchManager.getDocsChangeDate(SearchManager.java:945)
at org.fao.geonet.kernel.DataManager.init(DataManager.java:346)
at org.fao.geonet.Geonetwork.start(Geonetwork.java:345)
at jeeves.server.JeevesEngine.initAppHandler(JeevesEngine.java:448)
at jeeves.server.JeevesEngine.init(JeevesEngine.java:191)
at jeeves.server.sources.http.JeevesServlet.init(JeevesServlet.java:84)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1238)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1151)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1038)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4997)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5289)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:725)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:701)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:939)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1812)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:724)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.openIndexWriter(DirectoryTaxonomyWriter.java:265)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:192)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:315)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:332)
at org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.init(TaxonomyIndexTracker.java:69)
at org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.(TaxonomyIndexTracker.java:63)
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.lazyInit(LuceneIndexLanguageTracker.java:115)
… 26 more

This is also shown in catalina.out. At this point, the URL has dropped back to the default node rather than the new node. At this point I can’t sign in to either node. I have tried renaming the lucene directory since that’s reported in the error message, and the only way I can sign in to the default node is to remove the second node from config-node and restart. The existing records get reindexed and everything works OK.

What else should I be doing to make this work?

Thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Hi Jo

I think no many people is using that feature. I know only GeoSource (http://www.geosource.fr/), but their documentation points to the official documentation in GeoNetwork, that I guess you already checked.

The first error you pointed seem related to the Lucene index and the 2on to the database, not sure if are related.

I have never try it, will check to give a try. But would be good if any of the developers/maintainers of this feature can provide some support to your request.

Regards,
Jose García

···

On Tue, Oct 11, 2016 at 10:34 AM, Jo Cook <jocook@anonymised.com> wrote:

Hi All,

Some further research on this- and two issues that might be related. The errors I’m seeing seem to be related to the Lucene Directory, which is currently specified in the tomcat options. My databases are both postgresql, and I’m seeing a lot of errors in the log files related to postgresql:

2016-10-10 14:56:18,131 DEBUG [org.springframework.orm.jpa.JpaTransactionManager] - Resuming suspended transaction after completion of inner transaction
2016-10-10 14:56:18,132 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Setting JDBC Connection [jdbc:postgresql://[myhost]:5432/[mydatabase], UserName=geonetwork3, PostgreSQL Native Driver] read-only
2016-10-10 14:56:18,132 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Could not set JDBC Connection read-only
org.postgresql.util.PSQLException: Cannot change transaction read-only property in the middle of a transaction.

Could these be related to the multinode errors I’m seeing?

Has anyone successfully set up a Geonetwork 3.2.x multinode server, with postgresql as the back end database that could give me some pointers?

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.


Check out the vibrant tech community on one of the world’s most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot


GeoNetwork-devel mailing list
GeoNetwork-devel@…537…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

On Fri, Oct 7, 2016 at 3:03 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’m forwarding this to the dev list to see if it gets a response- I’m really keen to get Multinode Geonetwork working and am happy to test/expand documentation if necessary.

Thanks

Jo

---------- Forwarded message ----------
From: Jo Cook <jocook@anonymised.com>
Date: Thu, Oct 6, 2016 at 4:04 PM
Subject: Multinode Geonetwork
To: “geonetwork-users@anonymised.comforge.net” <geonetwork-users@anonymised.comforge.net>

Hi Developers,

I’m testing Geonetwork 3.2.x, specifically in multinode format. I’ve configured a second node using these instructions: http://geonetwork-opensource.org/manuals/trunk/eng/users/maintainer-guide/installing/using-multinode-mode.html

I’ve done this manually (ie adding a node to WEB-INF/web.xml and a node in config-node copied from the example xml in the link above), and also using the shell script.

The second database (postgres) is populated on a tomcat restart and I can visit the home page for the second node, which shows no metadata, as expected. When I sign in I get the following error in the browser:

Raised exception while starting the application. Fix the error and restart.

Message
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
Handler
org.fao.geonet.Geonetwork
Exception
java.lang.RuntimeException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
Stack
java.lang.RuntimeException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.lazyInit(LuceneIndexLanguageTracker.java:124)
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.acquire(LuceneIndexLanguageTracker.java:187)
at org.fao.geonet.kernel.search.SearchManager.getIndexReader(SearchManager.java:1210)
at org.fao.geonet.kernel.search.SearchManager.getNewIndexReader(SearchManager.java:1215)
at org.fao.geonet.kernel.search.SearchManager.getDocsChangeDate(SearchManager.java:945)
at org.fao.geonet.kernel.DataManager.init(DataManager.java:346)
at org.fao.geonet.Geonetwork.start(Geonetwork.java:345)
at jeeves.server.JeevesEngine.initAppHandler(JeevesEngine.java:448)
at jeeves.server.JeevesEngine.init(JeevesEngine.java:191)
at jeeves.server.sources.http.JeevesServlet.init(JeevesServlet.java:84)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1238)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1151)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1038)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4997)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5289)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:725)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:701)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:939)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1812)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:724)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.openIndexWriter(DirectoryTaxonomyWriter.java:265)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:192)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:315)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:332)
at org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.init(TaxonomyIndexTracker.java:69)
at org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.(TaxonomyIndexTracker.java:63)
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.lazyInit(LuceneIndexLanguageTracker.java:115)
… 26 more

This is also shown in catalina.out. At this point, the URL has dropped back to the default node rather than the new node. At this point I can’t sign in to either node. I have tried renaming the lucene directory since that’s reported in the error message, and the only way I can sign in to the default node is to remove the second node from config-node and restart. The existing records get reindexed and everything works OK.

What else should I be doing to make this work?

Thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

Hi,

···

2016-10-11 10:34 GMT+02:00 Jo Cook <jocook@anonymised.com>:

Hi All,

Some further research on this- and two issues that might be related. The errors I’m seeing seem to be related to the Lucene Directory, which is currently specified in the tomcat options. My databases are both postgresql, and I’m seeing a lot of errors in the log files related to postgresql:

2016-10-10 14:56:18,131 DEBUG [org.springframework.orm.jpa.JpaTransactionManager] - Resuming suspended transaction after completion of inner transaction
2016-10-10 14:56:18,132 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Setting JDBC Connection [jdbc:postgresql://[myhost]:5432/[mydatabase], UserName=geonetwork3, PostgreSQL Native Driver] read-only
2016-10-10 14:56:18,132 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Could not set JDBC Connection read-only
org.postgresql.util.PSQLException: Cannot change transaction read-only property in the middle of a transaction.

Could these be related to the multinode errors I’m seeing?

Has anyone successfully set up a Geonetwork 3.2.x multinode server, with postgresql as the back end database that could give me some pointers?

After a quick test on H2 it’s working fine. Not tested with postgres.

Francois

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.


Check out the vibrant tech community on one of the world’s most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot


GeoNetwork-devel mailing list
GeoNetwork-devel@…537…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

On Fri, Oct 7, 2016 at 3:03 PM, Jo Cook <jocook@anonymised.com…> wrote:

Hi Developers,

I’m forwarding this to the dev list to see if it gets a response- I’m really keen to get Multinode Geonetwork working and am happy to test/expand documentation if necessary.

Thanks

Jo

---------- Forwarded message ----------
From: Jo Cook <jocook@anonymised.com>
Date: Thu, Oct 6, 2016 at 4:04 PM
Subject: Multinode Geonetwork
To: “geonetwork-users@anonymised.com80…forge.net” <geonetwork-users@anonymised.comforge.net>

Hi Developers,

I’m testing Geonetwork 3.2.x, specifically in multinode format. I’ve configured a second node using these instructions: http://geonetwork-opensource.org/manuals/trunk/eng/users/maintainer-guide/installing/using-multinode-mode.html

I’ve done this manually (ie adding a node to WEB-INF/web.xml and a node in config-node copied from the example xml in the link above), and also using the shell script.

The second database (postgres) is populated on a tomcat restart and I can visit the home page for the second node, which shows no metadata, as expected. When I sign in I get the following error in the browser:

Raised exception while starting the application. Fix the error and restart.

Message
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
Handler
org.fao.geonet.Geonetwork
Exception
java.lang.RuntimeException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
Stack
java.lang.RuntimeException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.lazyInit(LuceneIndexLanguageTracker.java:124)
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.acquire(LuceneIndexLanguageTracker.java:187)
at org.fao.geonet.kernel.search.SearchManager.getIndexReader(SearchManager.java:1210)
at org.fao.geonet.kernel.search.SearchManager.getNewIndexReader(SearchManager.java:1215)
at org.fao.geonet.kernel.search.SearchManager.getDocsChangeDate(SearchManager.java:945)
at org.fao.geonet.kernel.DataManager.init(DataManager.java:346)
at org.fao.geonet.Geonetwork.start(Geonetwork.java:345)
at jeeves.server.JeevesEngine.initAppHandler(JeevesEngine.java:448)
at jeeves.server.JeevesEngine.init(JeevesEngine.java:191)
at jeeves.server.sources.http.JeevesServlet.init(JeevesServlet.java:84)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1238)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1151)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1038)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4997)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5289)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:725)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:701)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:939)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1812)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:724)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.openIndexWriter(DirectoryTaxonomyWriter.java:265)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:192)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:315)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:332)
at org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.init(TaxonomyIndexTracker.java:69)
at org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.(TaxonomyIndexTracker.java:63)
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.lazyInit(LuceneIndexLanguageTracker.java:115)
… 26 more

This is also shown in catalina.out. At this point, the URL has dropped back to the default node rather than the new node. At this point I can’t sign in to either node. I have tried renaming the lucene directory since that’s reported in the error message, and the only way I can sign in to the default node is to remove the second node from config-node and restart. The existing records get reindexed and everything works OK.

What else should I be doing to make this work?

Thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Hi Francois,

Thanks for checking- I can confirm that in a local install, running under jetty, it works using the default h2 database, but also that it works using postgresql. I’ll do some comparisons between my two installations to see if I can isolate the difference.

Regards

Jo

···

On Tue, Oct 11, 2016 at 12:24 PM, Francois Prunayre <fx.prunayre@anonymised.com> wrote:

Hi,

2016-10-11 10:34 GMT+02:00 Jo Cook <jocook@anonymised.com>:

Hi All,

Some further research on this- and two issues that might be related. The errors I’m seeing seem to be related to the Lucene Directory, which is currently specified in the tomcat options. My databases are both postgresql, and I’m seeing a lot of errors in the log files related to postgresql:

2016-10-10 14:56:18,131 DEBUG [org.springframework.orm.jpa.JpaTransactionManager] - Resuming suspended transaction after completion of inner transaction
2016-10-10 14:56:18,132 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Setting JDBC Connection [jdbc:postgresql://[myhost]:5432/[mydatabase], UserName=geonetwork3, PostgreSQL Native Driver] read-only
2016-10-10 14:56:18,132 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Could not set JDBC Connection read-only
org.postgresql.util.PSQLException: Cannot change transaction read-only property in the middle of a transaction.

Could these be related to the multinode errors I’m seeing?

Has anyone successfully set up a Geonetwork 3.2.x multinode server, with postgresql as the back end database that could give me some pointers?

After a quick test on H2 it’s working fine. Not tested with postgres.

Francois

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.


Check out the vibrant tech community on one of the world’s most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot


GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comorge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

On Fri, Oct 7, 2016 at 3:03 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’m forwarding this to the dev list to see if it gets a response- I’m really keen to get Multinode Geonetwork working and am happy to test/expand documentation if necessary.

Thanks

Jo

---------- Forwarded message ----------
From: Jo Cook <jocook@anonymised.com>
Date: Thu, Oct 6, 2016 at 4:04 PM
Subject: Multinode Geonetwork
To: “geonetwork-users@anonymised.com.forge.net” <geonetwork-users@anonymised.comforge.net>

Hi Developers,

I’m testing Geonetwork 3.2.x, specifically in multinode format. I’ve configured a second node using these instructions: http://geonetwork-opensource.org/manuals/trunk/eng/users/maintainer-guide/installing/using-multinode-mode.html

I’ve done this manually (ie adding a node to WEB-INF/web.xml and a node in config-node copied from the example xml in the link above), and also using the shell script.

The second database (postgres) is populated on a tomcat restart and I can visit the home page for the second node, which shows no metadata, as expected. When I sign in I get the following error in the browser:

Raised exception while starting the application. Fix the error and restart.

Message
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
Handler
org.fao.geonet.Geonetwork
Exception
java.lang.RuntimeException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
Stack
java.lang.RuntimeException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.lazyInit(LuceneIndexLanguageTracker.java:124)
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.acquire(LuceneIndexLanguageTracker.java:187)
at org.fao.geonet.kernel.search.SearchManager.getIndexReader(SearchManager.java:1210)
at org.fao.geonet.kernel.search.SearchManager.getNewIndexReader(SearchManager.java:1215)
at org.fao.geonet.kernel.search.SearchManager.getDocsChangeDate(SearchManager.java:945)
at org.fao.geonet.kernel.DataManager.init(DataManager.java:346)
at org.fao.geonet.Geonetwork.start(Geonetwork.java:345)
at jeeves.server.JeevesEngine.initAppHandler(JeevesEngine.java:448)
at jeeves.server.JeevesEngine.init(JeevesEngine.java:191)
at jeeves.server.sources.http.JeevesServlet.init(JeevesServlet.java:84)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1238)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1151)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1038)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4997)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5289)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:725)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:701)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:939)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1812)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:724)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.openIndexWriter(DirectoryTaxonomyWriter.java:265)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:192)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:315)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:332)
at org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.init(TaxonomyIndexTracker.java:69)
at org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.(TaxonomyIndexTracker.java:63)
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.lazyInit(LuceneIndexLanguageTracker.java:115)
… 26 more

This is also shown in catalina.out. At this point, the URL has dropped back to the default node rather than the new node. At this point I can’t sign in to either node. I have tried renaming the lucene directory since that’s reported in the error message, and the only way I can sign in to the default node is to remove the second node from config-node and restart. The existing records get reindexed and everything works OK.

What else should I be doing to make this work?

Thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Hi Francois,

By removing the explicit reference to the lucene directory in my tomcat configuration I have managed to get a little bit further with my main (non-local) installation and now no longer see the lucene error in my original email. Both sites now load, and I can sign in to my original (srv) node. However if I switch to my new node then I can’t sign in. I’ve verified that the admin user has the default admin password, but signing in with the new node returns the login failure page for the default node. I have tested this in chrome incognito and firefox with cookies cleared. I’ve tried clearing out the wro4j files from the data directory as well.

The log files show this:

2016-10-11 14:23:16,857 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/loc/api/xml/strings.xmlbut failed so falling back to default language
2016-10-11 14:23:16,901 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139/loc/api/labels.xmlbut failed so falling back to default language
2016-10-11 14:23:16,907 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139/loc/api/codelists.xmlbut failed so falling back to default language
2016-10-11 14:23:16,913 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139/loc/api/strings.xmlbut failed so falling back to default language
2016-10-11 14:23:16,915 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/dublin-core/loc/api/labels.xmlbut failed so falling back to default language
2016-10-11 14:23:16,919 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/dublin-core/loc/api/codelists.xmlbut failed so falling back to default language
2016-10-11 14:23:16,922 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/dublin-core/loc/api/strings.xmlbut failed so falling back to default language
2016-10-11 14:23:16,925 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/csw-record/loc/api/labels.xmlbut failed so falling back to default language
2016-10-11 14:23:16,928 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/csw-record/loc/api/codelists.xmlbut failed so falling back to default language
2016-10-11 14:23:16,931 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/csw-record/loc/api/strings.xmlbut failed so falling back to default language
2016-10-11 14:23:16,935 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19110/loc/api/labels.xmlbut failed so falling back to default language
2016-10-11 14:23:16,938 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19110/loc/api/codelists.xmlbut failed so falling back to default language
2016-10-11 14:23:16,941 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19110/loc/api/strings.xmlbut failed so falling back to default language
2016-10-11 14:23:16,944 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139.gemini22/loc/api/labels.xmlbut failed so falling back to default language
2016-10-11 14:23:16,949 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139.gemini22/loc/api/codelists.xmlbut failed so falling back to default language
2016-10-11 14:23:16,953 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139.gemini22/loc/api/strings.xmlbut failed so falling back to default language
2016-10-11 14:23:16,956 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/loc/api/xml/i18n.xmlbut failed so falling back to default language
2016-10-11 14:23:17,089 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/loc/api/xml/strings.xmlbut failed so falling back to default language

2016-10-11 14:23:19,323 ERROR [geonetwork.wro4j] - Error occurred during a wro4j request handling
ro.isdc.wro.WroRuntimeException: Cannot build valid CacheKey from request: /geonetwork/static/bootstrap-tagsinput.min.js.map
at ro.isdc.wro.manager.ResourceBundleProcessor.getSafeCacheKey(ResourceBundleProcessor.java:111)
at ro.isdc.wro.manager.ResourceBundleProcessor.serveProcessedBundle(ResourceBundleProcessor.java:61)
at ro.isdc.wro.manager.WroManager.process(WroManager.java:159)
at ro.isdc.wro.http.WroFilter.processRequest(WroFilter.java:335)
at ro.isdc.wro.http.WroFilter.doFilter(WroFilter.java:289)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:381)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:186)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
at jeeves.config.springutil.JeevesDelegatingFilterProxy.doFilter(JeevesDelegatingFilterProxy.java:104)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1095)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)

Does anyone have any pointers for me?

Thanks

Jo

···

On Tue, Oct 11, 2016 at 1:48 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Francois,

Thanks for checking- I can confirm that in a local install, running under jetty, it works using the default h2 database, but also that it works using postgresql. I’ll do some comparisons between my two installations to see if I can isolate the difference.

Regards

Jo

On Tue, Oct 11, 2016 at 12:24 PM, Francois Prunayre <fx.prunayre@anonymised.com> wrote:

Hi,

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

2016-10-11 10:34 GMT+02:00 Jo Cook <jocook@anonymised.com>:

Hi All,

Some further research on this- and two issues that might be related. The errors I’m seeing seem to be related to the Lucene Directory, which is currently specified in the tomcat options. My databases are both postgresql, and I’m seeing a lot of errors in the log files related to postgresql:

2016-10-10 14:56:18,131 DEBUG [org.springframework.orm.jpa.JpaTransactionManager] - Resuming suspended transaction after completion of inner transaction
2016-10-10 14:56:18,132 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Setting JDBC Connection [jdbc:postgresql://[myhost]:5432/[mydatabase], UserName=geonetwork3, PostgreSQL Native Driver] read-only
2016-10-10 14:56:18,132 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Could not set JDBC Connection read-only
org.postgresql.util.PSQLException: Cannot change transaction read-only property in the middle of a transaction.

Could these be related to the multinode errors I’m seeing?

Has anyone successfully set up a Geonetwork 3.2.x multinode server, with postgresql as the back end database that could give me some pointers?

After a quick test on H2 it’s working fine. Not tested with postgres.

Francois

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.


Check out the vibrant tech community on one of the world’s most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot


GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comorge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

On Fri, Oct 7, 2016 at 3:03 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’m forwarding this to the dev list to see if it gets a response- I’m really keen to get Multinode Geonetwork working and am happy to test/expand documentation if necessary.

Thanks

Jo

---------- Forwarded message ----------
From: Jo Cook <jocook@anonymised.com>
Date: Thu, Oct 6, 2016 at 4:04 PM
Subject: Multinode Geonetwork
To: “geonetwork-users@anonymised.comforge.net” <geonetwork-users@anonymised.comforge.net>

Hi Developers,

I’m testing Geonetwork 3.2.x, specifically in multinode format. I’ve configured a second node using these instructions: http://geonetwork-opensource.org/manuals/trunk/eng/users/maintainer-guide/installing/using-multinode-mode.html

I’ve done this manually (ie adding a node to WEB-INF/web.xml and a node in config-node copied from the example xml in the link above), and also using the shell script.

The second database (postgres) is populated on a tomcat restart and I can visit the home page for the second node, which shows no metadata, as expected. When I sign in I get the following error in the browser:

Raised exception while starting the application. Fix the error and restart.

Message
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
Handler
org.fao.geonet.Geonetwork
Exception
java.lang.RuntimeException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
Stack
java.lang.RuntimeException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.lazyInit(LuceneIndexLanguageTracker.java:124)
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.acquire(LuceneIndexLanguageTracker.java:187)
at org.fao.geonet.kernel.search.SearchManager.getIndexReader(SearchManager.java:1210)
at org.fao.geonet.kernel.search.SearchManager.getNewIndexReader(SearchManager.java:1215)
at org.fao.geonet.kernel.search.SearchManager.getDocsChangeDate(SearchManager.java:945)
at org.fao.geonet.kernel.DataManager.init(DataManager.java:346)
at org.fao.geonet.Geonetwork.start(Geonetwork.java:345)
at jeeves.server.JeevesEngine.initAppHandler(JeevesEngine.java:448)
at jeeves.server.JeevesEngine.init(JeevesEngine.java:191)
at jeeves.server.sources.http.JeevesServlet.init(JeevesServlet.java:84)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1238)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1151)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1038)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4997)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5289)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:725)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:701)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:939)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1812)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:724)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.openIndexWriter(DirectoryTaxonomyWriter.java:265)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:192)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:315)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:332)
at org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.init(TaxonomyIndexTracker.java:69)
at org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.(TaxonomyIndexTracker.java:63)
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.lazyInit(LuceneIndexLanguageTracker.java:115)
… 26 more

This is also shown in catalina.out. At this point, the URL has dropped back to the default node rather than the new node. At this point I can’t sign in to either node. I have tried renaming the lucene directory since that’s reported in the error message, and the only way I can sign in to the default node is to remove the second node from config-node and restart. The existing records get reindexed and everything works OK.

What else should I be doing to make this work?

Thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Hi All,

So looking at this error- it would seem as if it’s not picking up the default language, as it is trying to find ./loc/api/* rather than ./loc/eng/*. I have been through the log files and can’t see any errors that relate to it not being able to find the language, and I’m definitely calling http://url/geonetwork/node/eng in both cases. I can see that the error comes from this file https://github.com/geonetwork/core-geonetwork/blob/3.2.x/core/src/main/java/jeeves/server/dispatchers/guiservices/XmlCacheManager.java but I have no idea where to start with fixing the problem!

Thanks

Jo

···

On Tue, Oct 11, 2016 at 3:26 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Francois,

By removing the explicit reference to the lucene directory in my tomcat configuration I have managed to get a little bit further with my main (non-local) installation and now no longer see the lucene error in my original email. Both sites now load, and I can sign in to my original (srv) node. However if I switch to my new node then I can’t sign in. I’ve verified that the admin user has the default admin password, but signing in with the new node returns the login failure page for the default node. I have tested this in chrome incognito and firefox with cookies cleared. I’ve tried clearing out the wro4j files from the data directory as well.

The log files show this:

2016-10-11 14:23:16,857 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/loc/api/xml/strings.xmlbut failed so falling back to default language
2016-10-11 14:23:16,901 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139/loc/api/labels.xmlbut failed so falling back to default language
2016-10-11 14:23:16,907 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139/loc/api/codelists.xmlbut failed so falling back to default language
2016-10-11 14:23:16,913 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139/loc/api/strings.xmlbut failed so falling back to default language
2016-10-11 14:23:16,915 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/dublin-core/loc/api/labels.xmlbut failed so falling back to default language
2016-10-11 14:23:16,919 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/dublin-core/loc/api/codelists.xmlbut failed so falling back to default language
2016-10-11 14:23:16,922 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/dublin-core/loc/api/strings.xmlbut failed so falling back to default language
2016-10-11 14:23:16,925 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/csw-record/loc/api/labels.xmlbut failed so falling back to default language
2016-10-11 14:23:16,928 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/csw-record/loc/api/codelists.xmlbut failed so falling back to default language
2016-10-11 14:23:16,931 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/csw-record/loc/api/strings.xmlbut failed so falling back to default language
2016-10-11 14:23:16,935 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19110/loc/api/labels.xmlbut failed so falling back to default language
2016-10-11 14:23:16,938 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19110/loc/api/codelists.xmlbut failed so falling back to default language
2016-10-11 14:23:16,941 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19110/loc/api/strings.xmlbut failed so falling back to default language
2016-10-11 14:23:16,944 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139.gemini22/loc/api/labels.xmlbut failed so falling back to default language
2016-10-11 14:23:16,949 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139.gemini22/loc/api/codelists.xmlbut failed so falling back to default language
2016-10-11 14:23:16,953 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139.gemini22/loc/api/strings.xmlbut failed so falling back to default language
2016-10-11 14:23:16,956 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/loc/api/xml/i18n.xmlbut failed so falling back to default language
2016-10-11 14:23:17,089 ERROR [jeeves.resources] - Error cloning the cached data. Attempted to get: /var/lib/tomcat8/webapps/geonetwork/loc/api/xml/strings.xmlbut failed so falling back to default language

2016-10-11 14:23:19,323 ERROR [geonetwork.wro4j] - Error occurred during a wro4j request handling
ro.isdc.wro.WroRuntimeException: Cannot build valid CacheKey from request: /geonetwork/static/bootstrap-tagsinput.min.js.map
at ro.isdc.wro.manager.ResourceBundleProcessor.getSafeCacheKey(ResourceBundleProcessor.java:111)
at ro.isdc.wro.manager.ResourceBundleProcessor.serveProcessedBundle(ResourceBundleProcessor.java:61)
at ro.isdc.wro.manager.WroManager.process(WroManager.java:159)
at ro.isdc.wro.http.WroFilter.processRequest(WroFilter.java:335)
at ro.isdc.wro.http.WroFilter.doFilter(WroFilter.java:289)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:381)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:186)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
at jeeves.config.springutil.JeevesDelegatingFilterProxy.doFilter(JeevesDelegatingFilterProxy.java:104)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1095)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)

Does anyone have any pointers for me?

Thanks

Jo

On Tue, Oct 11, 2016 at 1:48 PM, Jo Cook <jocook@anonymised.com036…> wrote:

Hi Francois,

Thanks for checking- I can confirm that in a local install, running under jetty, it works using the default h2 database, but also that it works using postgresql. I’ll do some comparisons between my two installations to see if I can isolate the difference.

Regards

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

On Tue, Oct 11, 2016 at 12:24 PM, Francois Prunayre <fx.prunayre@anonymised.com> wrote:

Hi,

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

2016-10-11 10:34 GMT+02:00 Jo Cook <jocook@anonymised.com>:

Hi All,

Some further research on this- and two issues that might be related. The errors I’m seeing seem to be related to the Lucene Directory, which is currently specified in the tomcat options. My databases are both postgresql, and I’m seeing a lot of errors in the log files related to postgresql:

2016-10-10 14:56:18,131 DEBUG [org.springframework.orm.jpa.JpaTransactionManager] - Resuming suspended transaction after completion of inner transaction
2016-10-10 14:56:18,132 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Setting JDBC Connection [jdbc:postgresql://[myhost]:5432/[mydatabase], UserName=geonetwork3, PostgreSQL Native Driver] read-only
2016-10-10 14:56:18,132 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Could not set JDBC Connection read-only
org.postgresql.util.PSQLException: Cannot change transaction read-only property in the middle of a transaction.

Could these be related to the multinode errors I’m seeing?

Has anyone successfully set up a Geonetwork 3.2.x multinode server, with postgresql as the back end database that could give me some pointers?

After a quick test on H2 it’s working fine. Not tested with postgres.

Francois

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.


Check out the vibrant tech community on one of the world’s most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot


GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comorge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

On Fri, Oct 7, 2016 at 3:03 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’m forwarding this to the dev list to see if it gets a response- I’m really keen to get Multinode Geonetwork working and am happy to test/expand documentation if necessary.

Thanks

Jo

---------- Forwarded message ----------
From: Jo Cook <jocook@anonymised.com>
Date: Thu, Oct 6, 2016 at 4:04 PM
Subject: Multinode Geonetwork
To: “geonetwork-users@anonymised.comforge.net” <geonetwork-users@anonymised.comforge.net>

Hi Developers,

I’m testing Geonetwork 3.2.x, specifically in multinode format. I’ve configured a second node using these instructions: http://geonetwork-opensource.org/manuals/trunk/eng/users/maintainer-guide/installing/using-multinode-mode.html

I’ve done this manually (ie adding a node to WEB-INF/web.xml and a node in config-node copied from the example xml in the link above), and also using the shell script.

The second database (postgres) is populated on a tomcat restart and I can visit the home page for the second node, which shows no metadata, as expected. When I sign in I get the following error in the browser:

Raised exception while starting the application. Fix the error and restart.

Message
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
Handler
org.fao.geonet.Geonetwork
Exception
java.lang.RuntimeException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
Stack
java.lang.RuntimeException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.lazyInit(LuceneIndexLanguageTracker.java:124)
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.acquire(LuceneIndexLanguageTracker.java:187)
at org.fao.geonet.kernel.search.SearchManager.getIndexReader(SearchManager.java:1210)
at org.fao.geonet.kernel.search.SearchManager.getNewIndexReader(SearchManager.java:1215)
at org.fao.geonet.kernel.search.SearchManager.getDocsChangeDate(SearchManager.java:945)
at org.fao.geonet.kernel.DataManager.init(DataManager.java:346)
at org.fao.geonet.Geonetwork.start(Geonetwork.java:345)
at jeeves.server.JeevesEngine.initAppHandler(JeevesEngine.java:448)
at jeeves.server.JeevesEngine.init(JeevesEngine.java:191)
at jeeves.server.sources.http.JeevesServlet.init(JeevesServlet.java:84)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1238)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1151)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1038)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4997)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5289)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:725)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:701)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:939)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1812)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/geonetwork/lucene_dir/taxonomy/write.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:724)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.openIndexWriter(DirectoryTaxonomyWriter.java:265)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:192)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:315)
at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:332)
at org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.init(TaxonomyIndexTracker.java:69)
at org.fao.geonet.kernel.search.index.TaxonomyIndexTracker.(TaxonomyIndexTracker.java:63)
at org.fao.geonet.kernel.search.index.LuceneIndexLanguageTracker.lazyInit(LuceneIndexLanguageTracker.java:115)
… 26 more

This is also shown in catalina.out. At this point, the URL has dropped back to the default node rather than the new node. At this point I can’t sign in to either node. I have tried renaming the lucene directory since that’s reported in the error message, and the only way I can sign in to the default node is to remove the second node from config-node and restart. The existing records get reindexed and everything works OK.

What else should I be doing to make this work?

Thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek