Hi Jo,
Not sure what is wrong, by DEBUG mode I was thinking from an IDE so you can check username and password are set properly.
Locally, I’m running a secured elastic instance for a month with no problem with version 4. I also deployed it with 7.11 & 7.12.
Testing the coming release of 4.0.4 on docker, what I have:
- with a wrong_user Elastic report wrong auth and the message is displayed in the UI
Unable to revive connection to http://192.168.1.102:9200. Error is Elasticsearch exception [type=security_exception, reason=unable to authenticate user [wrong_user] for REST request [/_cluster/health?master_timeout=30s&level=cluster&timeout=30s]]
- with wrong password
in log at startup
Could not connect to index ‘gn-records’. Error is method [HEAD], host [http://192.168.1.102:9200], URI [/gn-records?ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=false], status line [HTTP/1.1 401 Unauthorized]. Is the index server up and running?
in UI
Unable to revive connection to http://192.168.1.102:9200. Error is Elasticsearch exception [type=security_exception, reason=unable to authenticate user [elastic] for REST request [/_cluster/health?master_timeout=30s&level=cluster&timeout=30s]]
- with correct username, looks ok.
Started with the main ‘elastic’ user:
docker run -p 8080:8080
-e “ES_HOST=192.168.1.102”
-e “ES_PORT=9200”
-e “ES_PROTOCOL=http”
-e “ES_USERNAME=elastic”
-e “ES_PASSWORD=aaaaaa”
-e “ES_INDEX_RECORDS=gn-records”
-e GEONETWORK_DB_TYPE=postgres
-e GEONETWORK_DB_HOST=192.168.1.102
-e GEONETWORK_DB_PORT=5432
-e GEONETWORK_DB_USERNAME=www-data
-e GEONETWORK_DB_PASSWORD=www-data
-e GEONETWORK_DB_NAME=gn
geonetwork:4.0.4
elastic.yml
xpack.ml.enabled: false
xpack.security.enabled: true
network.host: 0.0.0.0
node.master: true
node.data: true
transport.host: localhost
transport.tcp.port: 9300
So 4.0.x should be fine regarding authentication. Not 100% sure about 3.x.
HTH.
Francois
Le jeu. 29 avr. 2021 à 13:36, Jo Cook <jocook@anonymised.com> a écrit :
Hi Francois,
I’ve also tested a docker version, with the GeoNetwork 4.0.3 image and ElasticSearch 7.12.1 and I’m still not getting anywhere. I’ve stripped my docker-compose file back to basics so there’s nothing custom, and I’ve added a modified config.properties with the es.username and es.password set, but GeoNetwork still can’t connect:
uninitialized
Unable to revive connection to http://elasticsearch:9200. Error is Elasticsearch exception [type=security_exception, reason=missing authentication credentials for REST request [/_cluster/health?master_timeout=30s&level=cluster&timeout=30s]]
I’ve set the users gn_user, gn_admin and anonymous as in https://github.com/geonetwork/core-geonetwork/pull/5623/files#
Thanks
Jo
On Thu, Apr 29, 2021 at 10:17 AM Jo Cook <jocook@anonymised.com> wrote:
Hi Francois,
/monitor/healthcheck reports a connection error:
! IndexHealthCheck: ERROR
! Index storing records is not available currently. This component is only required if you use WFS features indexing and dashboards.
I’m not quite sure what you mean by running it in debug mode to check whether the values are set. What I’m doing at the moment is running it using the following command:
mvn jetty:run -Penv-dev -Pes
Then I’ve got the logs set in index mode.
When I run /monitor/healthcheck I get the following error:
2021-04-29 10:13:41,216 WARN [geonetwork.index] - EsClient bulkRequest: {“root_cause”:[{“type”:“security_exception”,“reason”:“missing authentication credentials for REST request [/gn-searchlogs/_bulk]”,“header”:{“WWW-Authenticate”:“Basic realm="security" charset="UTF-8"”}}],“type”:“security_exception”,“reason”:“missing authentication credentials for REST request [/gn-searchlogs/_bulk]”,“header”:{“WWW-Authenticate”:“Basic realm="security" charset="UTF-8"”}}
I’m not sure how to check whether the username and password are actually set? I have also tried this passing the environment variables es.username and es.password as part of the jetty:run command, I don’t know whether that was worth doing and it didn’t make any difference anyway…
Thanks
Jo
On Thu, Apr 29, 2021 at 6:58 AM Francois Prunayre <fx.prunayre@anonymised.com> wrote:
Jo, and what is the status of the remote index in admin > Stat & Status ?
https://github.com/geonetwork/core-geonetwork/blob/3.10.x/healthmonitor/src/main/java/org/fao/geonet/monitor/health/IndexHealthCheck.java
If connection fails it should also report an error ?
Francois
Le mer. 28 avr. 2021 à 18:31, Jo Cook <jocook@anonymised.com> a écrit :
Hi Francois,
I’ll try the debug approach.
I’m using a customised fork of 3.10 but the commit ID for the latest core-geonetwork/3.10.x commit is 1db3eebe97c9b932228a8a00d3d706d35c3ff463 (31st March). I’m pretty sure that the custom changes won’t have made any difference.
Thanks
Jo
On Wed, Apr 28, 2021 at 5:05 PM Francois Prunayre <fx.prunayre@anonymised.com…> wrote:
The connection is define here
https://github.com/geonetwork/core-geonetwork/blob/main/index/src/main/java/org/fao/geonet/index/es/EsRestClient.java#L146
it you’ve the chance to start it in debug mode to check that properties are correct ?
Which 3.10 are you using? commit ID ?
Francois
Le mer. 28 avr. 2021 à 17:49, Jo Cook <jocook@anonymised.com> a écrit :
Hi Francois,
Thanks- I think the main difference I have is that I have just used the same username and password throughout for testing (after setting it using “elasticsearch-setup-passwords interactive”) rather than using the cURL commands to set the users. For testing everything is using the main “elastic” user so I think it should have full control? I can definitely load the indices and kibana can connect- it’s just geonetwork that can’t.
All the best
Jo
On Wed, Apr 28, 2021 at 4:28 PM Francois Prunayre <fx.prunayre@anonymised.com1…> wrote:
Hi Jo, it is supposed to work well.
I’m documenting it in more detail here https://github.com/geonetwork/core-geonetwork/pull/5623
Maybe in 3.x there is some config missing but would require to check it in more detail but I have some projects based on 3.x and 4 using secured cluster.
Cheers.
Francois
Le mer. 28 avr. 2021 à 17:24, Jo Cook <jocook@anonymised.com> a écrit :
Hi All,
I’m still struggling with this- and I’m wondering if it’s even possible to run GeoNetwork 3.10 against elasticsearch secured with a username and password? Can someone confirm for me?
Thanks
Jo
On Tue, Apr 20, 2021 at 10:39 AM Jo Cook <jocook@anonymised.com> wrote:
Hi All,
I’m trying to run GeoNetwork (3.10.x) against a elasticsearch secured with a username and password. I’ve tried with a locally built GeoNetwork both in jetty and using docker. GeoNetwork was definitely built with the es profile enabled. I have read https://www.geonetwork-opensource.org/manuals/4.0.x/en/install-guide/installing-index.html and have included the username and password in WEB-INF/config.properties.
Kibana can successfully connect to the authenticated instance, and I can load indices using cURL (so it’s not a read-only user) but when I try to index the records in GeoNetwork using “admin console/tools/re-index in a remote instance” I get the following error immediately:
There is an error with the ElasticSearch index. See the logs for details
IOException
Error during removal. Errors is ‘{“root_cause”:[{“type”:“security_exception”,“reason”:“missing authentication credentials for REST request [/gn-records/_delete_by_query]”,“header”:{“WWW-Authenticate”:“Basic realm="security" charset="UTF-8"”}}],“type”:“security_exception”,“reason”:“missing authentication credentials for REST request [/gn-records/_delete_by_query]”,“header”:{“WWW-Authenticate”:“Basic realm="security" charset="UTF-8"”}}’.
Could anyone help me with what I’m missing here? Do I need to set the elasticsearch credentials in GeoNetwork before building the war file?
Thanks
Jo
–
Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek
Please note that currently I do not work on Friday afternoons. For urgent responses at that time, please visit support.astuntechnology.com or phone our office on 01372 744009
–
Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek
Please note that currently I do not work on Friday afternoons. For urgent responses at that time, please visit support.astuntechnology.com or phone our office on 01372 744009
–
Sign up to our mailing list for updates on news, products, conferences, events and training
Astun Technology Ltd, 120 Manor Green Road, Epsom, Surrey, KT19 8LN, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech
iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal
Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
–
Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek
Please note that currently I do not work on Friday afternoons. For urgent responses at that time, please visit support.astuntechnology.com or phone our office on 01372 744009
–
Sign up to our mailing list for updates on news, products, conferences, events and training
Astun Technology Ltd, 120 Manor Green Road, Epsom, Surrey, KT19 8LN, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech
iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal
Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.
–
Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek
Please note that currently I do not work on Friday afternoons. For urgent responses at that time, please visit support.astuntechnology.com or phone our office on 01372 744009
–
Sign up to our mailing list for updates on news, products, conferences, events and training
Astun Technology Ltd, 120 Manor Green Road, Epsom, Surrey, KT19 8LN, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech
iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal
Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.
–
Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek
Please note that currently I do not work on Friday afternoons. For urgent responses at that time, please visit support.astuntechnology.com or phone our office on 01372 744009
–
Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek
Please note that currently I do not work on Friday afternoons. For urgent responses at that time, please visit support.astuntechnology.com or phone our office on 01372 744009
–
Sign up to our mailing list for updates on news, products, conferences, events and training
Astun Technology Ltd, 120 Manor Green Road, Epsom, Surrey, KT19 8LN, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech
iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal
Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.