Hi List,
I've been noticing heavy IO/application lockup when serving large shapefiles with geoserver 2.2.1/geotools 8.3 under very heavy load simulated with JMeter.
If the "Create spatial index if missing/outdated" option is left enabled, the whole system gets bogged down checking if indices were up-to-date. Disabling this option speeds things up massively up but prevents the index being used altogether for WFS getfeature queries filtering by featureid, so these queries take minutes instead of seconds to return.
Tracing the checkbox through the system, I ended up in the GeoTools project in the ShapeFileDataStoreFactory.java file. I can see the checkbox value being used as a switch to enable/disable using the shapefile index:
if (createIndex) {
store = new IndexedShapefileDataStore(url, namespace,
useMemoryMappedBuffer, cacheMemoryMaps, true, IndexType.QIX,
dbfCharset);
} else {
store = new ShapefileDataStore(url, namespace,
useMemoryMappedBuffer, cacheMemoryMaps, dbfCharset);
}
I'd really like to disable index updating in my production environment but I also want to be able to use shapefile indexes. I've developed a rough and ready git patch against geotools 8.3 to allow this (attached). The patch creates a new variable to always enable the index for local files. I could rework this to use the param class and be set in the map that gets sent to createNewDataStore() Is there any interest in this?
For GeoServer, the current checkbox label is a bit misleading, it took me a while to figure out why the indexes weren't being used. Perhaps it could be changed to say something like "enable spatial index and update if missing/outdate" or split into two options if the GeoTools code is changed to support it:
* Enable spatial index
* Create spatial index if missing/outdated
Would it possible to get this fix or something similar into GeoServer and Geotools?
Thanks,
Geoff
shapefile_index_switch.patch (1.75 KB)
