Error with Importer Plugin

Hi All,

I’m hitting an issue when trying to use the Importer plugin in Geoserver. It seems to have started from 2.21 onwards.
I currently have 2.23.6 running.

When I run the import process:

curl -u admin:password -XPOST -H "Content-type: application/json" \
    -d '{"import": {"targetWorkspace": {"workspace": {"name": "name"}}, "targetStore": {"dataStore": {"name": "datastore"}}, "data": {"type": "remote", "location": "file.shz"}}}' \
    https://localhost/geoserver/rest/imports

I’m greeted with an error:
Handler dispatch failed; nested exception is java.lang.NoClassDefFoundError: org/apache/commons/httpclient/util/URIUtil’

Could this be due to an upgrade of the HTTPClient to v4 and removal of v3?

Any assistance would be appreciated.

It does sound like a packaging issue, can you try a supported release please.

Thanks for the suggestion.
I have tried it with v 2.25.3.
I get the same error however.
I ensured to grab the same version importer package as well.

Just to add to this.

I have pulled down the 2.25.3 container and was able to replicate the issue.

Container setup:

docker run -it -p8080:8080 `
 --env INSTALL_EXTENSIONS=true `
 --env STABLE_EXTENSIONS="importer" `
 --mount type=bind,src=D:\goeserver,target=/opt/geoserver_data `
 docker.osgeo.org/geoserver:2.25.3

Did the following post call:
http://localhost:8080/geoserver/rest/imports

{"import": {"targetWorkspace": {"workspace": {"name": "ne"}}, "targetStore": {"dataStore": {"name": "NaturalEarth"}}, "data": {"type": "remote", "location": "https://data.gov.au/data/dataset/f9d2d353-ae14-45c0-9f59-d0ed2cb00b74/resource/a8479f7c-1682-4b28-9190-adbb589ebda3/download/managed_floodplain_flowmer_2022.zip"}}}

^^ Test dataset.

The error happened.

Handler dispatch failed; nested exception is java.lang.NoClassDefFoundError: org/apache/commons/httpclient/util/URIUtil

@jive Did you need more information on this issue?

Just seeing if I can get a confirmation / bump on this one.
This has caused issues in our production environment for a while now.

Thanks for testing with several versions. I just put out as request to test with nightly build ahead of release next week.

  • I do not see any class called “org.apache.httpclient.URIUtil” in the http client jar.
  • I cannot see GeoServer code calling “org.apache.httpclient.URIUtil”
  • I cannot see any GeoTools code calling “org.apache.httpclient.URIUtil”

So we need to find the stack trace from the logs. You supplied a screen snap - but I need to see the “caused by” information a bit further down. Something is odd

test setup

  1. Using your example to test the latest:
docker run -it -p8080:8080 \
 --env INSTALL_EXTENSIONS=true \
 --env STABLE_EXTENSIONS="importer" \
  --env CORS_ENABLED=true \
 docker.osgeo.org/geoserver:2.26.x
  1. We need an example that works with the default data directory.

    Working from Importer REST API examples — GeoServer 2.26.x User Manual

  2. Adjust import.json so we reference a local folder:

{
  "import": {
    "targetWorkspace": {
      "workspace": {
        "name": "ne"
      }
    },
    "data": {
      "type": "directory",
      "location": "/opt/geoserver_data/data/shapefiles"
    }
  }
}
  1. Adapting the exaple:
curl -v -u admin:geoserver -XPOST -H "Content-type: application/json" \
  -H "Accept: application/json" \
  -d @import.json \
  "http://localhost:8080/geoserver/rest/imports"
  1. And I get something …
...
< HTTP/1.1 201 
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< Location: http://localhost:8080/geoserver/rest/imports/2
< Content-Type: application/json
< Transfer-Encoding: chunked
< Date: Tue, 10 Sep 2024 02:52:21 GMT
< 
* Connection #0 to host localhost left intact
{"import":{"id":2,"href":"http://localhost:8080/geoserver/rest/imports/2","state":"PENDING","archive":false,"targetWorkspace":{"workspace":{"name":"ne","isolated":false,"dateCreated":"2022-08-21 15:10:28.234 UTC","dateModified":"2022-09-23 08:29:42.450 UTC"}},"data":{"type":"directory","format":"Shapefile","location":"/opt/geoserver_data/data/shapefiles","href":"http://localhost:8080/geoserver/rest/imports/2/data"},"tasks":[{"id":0,"href":"http://localhost:8080/geoserver/rest/imports/2/tasks/0","state":"READY"}]}}
  1. And I can see in the GUI that the import was created

reproducing your test

Your example listed “remote” which is intended to like download from a URL location … so it makes sense it would use http client. However your example listed file.shz - which is like a local file?

Trying it out on a public shape file download: https://www.naturalearthdata.com/
https://naciscdn.org/naturalearth/110m/cultural/ne_110m_admin_0_countries.zip

{
  "import": {
    "targetWorkspace": {
      "workspace": {
        "name": "ne"
      }
    },
    "data": {
      "type": "remote",
      "location": "/Users/jgarnett/Downloads/ne_110m_admin_0_countries.zip"
    }
  }
}

And I can finally see your error:

* upload completely sent off: 233 bytes
< HTTP/1.1 500 
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< Content-Type: text/plain
< Transfer-Encoding: chunked
< Date: Tue, 10 Sep 2024 02:59:18 GMT
< Connection: close
< 
* Closing connection
Handler dispatch failed; nested exception is java.lang.NoClassDefFoundError: org/apache/commons/httpclient/util/URIUtil

And the stack trace:

Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/util/URIUtil
	at org.apache.commons.vfs2.provider.URLFileName.getPathQueryEncoded(URLFileName.java:91)
	at org.apache.commons.vfs2.provider.URLFileName.getURIEncoded(URLFileName.java:161)
	at org.apache.commons.vfs2.provider.url.UrlFileObject.createURL(UrlFileObject.java:74)
	at org.apache.commons.vfs2.provider.url.UrlFileObject.doAttach(UrlFileObject.java:63)
	at org.apache.commons.vfs2.provider.AbstractFileObject.attach(AbstractFileObject.java:1505)
	at org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:489)
	at org.apache.commons.vfs2.provider.AbstractFileObject.exists(AbstractFileObject.java:477)
	at org.apache.commons.vfs2.provider.AbstractFileObject.copyFrom(AbstractFileObject.java:1013)
	at org.geoserver.importer.Directory.accept(Directory.java:395)
        ...
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.util.URIUtil
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1308)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1120)

found

So I think we have enough for a bug report … [GEOS-11533] - JIRA

  • The “virtual file system” thing, vfs2, seems to not be compatible with this version of httpclient?
  • Checking Commons VFS – Project Dependencies
  • Shows that this version 2.9.1 is compatible with httpclient 3.1 used by GeoServer

update: I was wrong “remote” does not seem to take a remote URL (https://naciscdn.org/naturalearth/110m/cultural/ne_110m_admin_0_countries.zip). It was previously used with the community module “REST resumable upload” to stage the data, and then import.

Whew!

Aside: if it has caused problems for a while in your production - please
report!

Contacting via forum is okay, but taking the time to make a reproducible
bug report in the issue tracker helps a lot. I really could not do anything
without the stack trace, and the steps to reproduce the problem.

Keep in mind that developers get to work on the code base, but do not get
to use GeoServer. We really depend on issues being reported, in some cases
issues are well know, or have workarounds, but are not reported :slight_smile:

Thanks for including docker and curl examples today.

Jody Garnett