For some Geoserver harvesters Geonetwork is blocked. I get a 403 error.
However, the same Geoserver can be consumed without problem from QGIS, and a simple GetCapabilities request from a web browser returns a valid XML document. After some troubleshooting with the aid of ChatGPT, we have concluded this:
The GeoServer is rejecting requests unless the User-Agent mimics a browser (like Mozilla/5.0).
GeoNetwork’s harvester uses a non-browser user-agent, so it’s blocked with a 403 Forbidden
Is it possible to set the User-Agent in Geonetwork?
Compare these two requests curl --ssl-no-revoke "https://mapsext.sgi.se/geoserver/wms?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetCapabilities"
I had a look in the GeoServer source code and did not find any specific check for Mozilla/5.0. I expect that the GeoServer is behind Apache or NginX configured to restrict user-agent.
In this case it does not make much sense as WMS is a general web service, not specifically intended for browser?
Looking at the welcome page there is an individual to contact: mats.oberg@sgi.se
Thanks. That was a good start. I was thinking that perhaps there was some variable in the Geonetwork harvester that could ‘pose’ it as Mozilla - or some other requestor. Asking providers about their server settings is a bit presumptuous, particularly since the wms service works as expected in all other aspects.
This approach led to a kind of solution. It appears that you need to make make the Java Virtual Machine (JVM) running GeoNetwork to appear as a standard browser User-Agent.
QGIS and GeoNetwork identify themselves differently when making web requests. QGIS sends a user-agent string identifying itself as a standard GIS client (e.g., Mozilla/5.0 QGIS/3.44). GeoNetwork, which runs on Java, often sends a default Java HTTP client user-agent (e.g., Java/1.8.0 or Apache-HttpClient). By creating a setenv.sh file in ../tomcat9/bin with this line: export CATALINA_OPTS=“$CATALINA_OPTS -Dhttp.agent=Mozilla/5.0”
the harvester poses as an ordinary web request coming from QGIS for example.
I have not checked the code for alternate solutions - and this is proved to work only at the location stated in TS.