The following is a list of items that came up during my development with the community importer.
The first part is bugs/issues with the importer, the second is geoserver bugs/issues, and the last is ideas for enhancements:
Import bugs/issues:
-
doc/markdown directory should be merged into sphinx docs and reviewed
-
revisit expand parameter in API - this provides more information on representations in the API - by expanding by default unless at the collection level
-
protect against a task being run again
-
prevent any localworkspace from being serialized
-
an unhandled exception in the importer may result in an error state but there is no way to track from the client side - for example, a runtime exception (non-user visible error) is hard to trace to the actual problem. Solution: introduce UUID for tracing the exception from the client to the server administrator.
-
change a layer name or nativeName results in an error when trying to get the attributes because the feature type no longer exists. Solution: put this at the task level, check for existing native name and throw if present. re-check again at import time
-
validate if direct import and the store format does not support a specific updateMode (or if it’s not implemented). probably not a good idea to support this on shapefile or other stores immediately. Longer term, would be nice to have but should 400 on such a request until supported.
-
fix Importer.run(ImportContext,ImportFilter,ProgressMonitor) to handle exceptions better - currently if the first task works but the second fails, state will be invalid
-
break implementation of Importer.doXXXImport into separate classes for testing to allow mocking and error-injection for verification of behavior
-
serialbinding works fine from my testing. remove use of xstreampersister to reduce permutations of configuration
-
for unifying (and constraining) docs and functionalty, perhaps the best approach is to note the philosophy of the API is to not duplicate anything from the other config API. Then, we enumerate the specific representation transitions supported in the docs, then ensure they’re covered. This came up for example with the way a LayerInfo was updated. The implementation was copying the (somehwat incomplete) FeatureInfo passed in ‘short-hand’ and basically overriding existing configuration. In reality, we only really want to allow changing the SRS (of the possible properties I could identify)
-
Any errors in the streaming response (see: ImportContextJSONFormat) are not caught because at that point the status has already been written. therefore the client just gets invalid JSON w/ a 200 status.
-
remove complexity of spring context - replace BeanResourceFinder with direct mapping
-
complete support for updateModes for database indirect import (probably should include truncating any cache as well)
Some general geoserver bugs/issues found during testing:
-
REST Config API - one can create a db datastore w/ invalid credentials and there is no way of getting feedback w/out reloading the catalog (otherwise it doesn’t seem to validate the new connection) ??
-
request to /geoserver/rest/imports just gives response /geoserver/rest response if importer not active (hard to test if the API is enabled/present)
-
Possible NPE when deleting resource after deleting layer?
at org.geoserver.catalog.ResourcePool$FeatureTypeCache.dispose(ResourcePool.java:1697)
at org.geoserver.catalog.ResourcePool$FeatureTypeCache.dispose(ResourcePool.java:1689)
at org.geoserver.catalog.ResourcePool$CatalogResourceCache.remove(ResourcePool.java:1668)
at org.geoserver.catalog.ResourcePool.clear(ResourcePool.java:1009)
Enhancements:
-
add capabilities/version endpoint (for example: /rest/importer/capabilities ) that describes supported formats, etc.
-
add support for testing projection from prj file before transferring data
-
if describing supported formats is too difficult, adding an endpoint for verifying if the provided files (by name only) can be processed before transferring any data.
-
support for fetching from remote URL
-
web-hooks for completion of import (to avoid polling the progress endpoint during async import)
–
Ian Schneider
Software Engineer | Boundless
ischneider@anonymised.com
