[Geoserver-devel] Shapefile mass importer for the stack installer recap

Hi all,
yesterday we chatted a bit about how the suite
installation process can include a step that allows
mass import of all the shapefiles contained in a directory.

I've attached the full logs if someone wants to
re-read them.

General importer spec
--------------------------------------------------
The user somehow specifies a directory containing
shapefiles, and the process is supposed to configure
them all in the GeoServer catalog automatically
setting:
1) the SRS
2) a default style
3) the bounding box
The bounding box is read from the shapefile header,
the default style is chosen from the geometry type
(should it be a copy of the default, or the
default itself? I.e., should each layer end up
with its own personal style or all polygon ones
share the same polygon.sld style?)

The SRS is tricky in that we need to come up
from an official EPSG code, the shapefiles might
come without a .prj from which we cannot
extract a EPSG code, or don't have a .prj at
all.

Options here:
a) the user specifies a default EPSG code for
    the cases in which we cannot determine a
    native EPSG code
b) the import process fails if no .prj is found,
    otherwise if it cannot determine an EPSG code
    it will just set 4326 and force a "reproject
    native to declared" srs handling

Option a) is nicer to the user (I have seen a whole
lot of shapefiles without .prj around) but requires
some decent UI to choose the default EPSG code
(something that allows to drill into the 3000+ available
codes)

Regardelss of the details, the moving parts needed
to pull this off are a tool to write the GeoServer
configruation plus the shapefile datastores (or better,
the directory datastore) so that we can gather the
shapefile structural information (we need attributes
too).
GeoServer itself has all of the above, anything
else will have to reinvent the wheel.
So it is suggested that whatever the UI is,
a running GeoServer ends up doing the heavy lifting.
This means we need to ensure GeoServer is running
once we start this import process.

Importer UI
------------------------------------------------

The importer UI should run during/right after
the installer did its job. It should also be
possible to run it multiple times to import
more than one set of shapefiles.

The latter should receive some more consideration imho,
my experience is that people tend to lump together
files by project. If they want to import multiple
projects we should consider allowing people to
keep them separate in GS as well (think organisation
issues and name conflicts).
A way to do so could be to ask them to provide
the URL of their site and a "project" name, and build
a workspace definition out of that.
For example, "www.google.com", "foo" would become
the "foo" workspace whose namespace is
"http://www.google.com/foo" and the shapefile "xyz"
would become the "foo:xyz" layer.

Besides this, the UI should provide a directory
chooser and an eventual EPSG code chooser (depends
on which of the SRS handling options we choose).

Clarification: the UI needs to choose a directory
being on the same machine GeoServer is running.

Options considered:
- an NSIS panel asking just for the directory,
   which ends up making a REST call to GeoServer
   passing the path, GeoServer does the rest.
   Pros:
   * native, integrated in the installer
   Cons:
   * can't be rerun after the install

- a Swing app asking the directory and eventually
   providing a simple SRS chooser, which ends
   up making a REST call to GeoServer passing
   the path and the SRS
   Pros:
   * can be rerun
   * we have a choice of widgets for both the tasks
   Cons:
   * a bit on the heavy side

- a Site Specific Browser running a pure HTML/JS
   that does ask for the local directory and the
   SRS
   Pros:
   * can be rerun
   * can be made to look very stylish
   Cons:
   * requires a js coder and some access to the
     local filesystem
   * we need a SSB solution that allows local FS
     access

- a Site Specific Browser running a Wicket wizard
   that is actually part of GeoServer itself
   Pros:
   * can be rerun, even from different hosts
   * can be integrated as part of the normal
     GeoServer experience
   Cons:
   * the directory chooser control is not part
     of the Wicket widgets, I made one in my
     spare time but it does not look good,
     so some design love is needed

If we go the SSB road, mind we need to include
a SSB in the installer, and we'll need a different
one for each platform (WebRunner works multiplatform,
but mind, it adds 8MB to the download).
I was wondering if we can't just open a plain browser
window (less stylish for sure).

Time wise, the quickest for me to develop is the
Swing one, I can make the Swing UI in probably around
half a day. The plain NSIS + rest call is quick too,
not sure we provide good feedback out of that
(a summary of how the import process did would be nice).

The one I like the best is the plain browser window
with the Wicket UI running in it.

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

(attachments)

InstallerTalk.txt (15.4 KB)

Andrea Aime ha scritto:

Hi all,
yesterday we chatted a bit about how the suite
installation process can include a step that allows
mass import of all the shapefiles contained in a directory.

Sorry, wrong mailing list. Anyways, you'll see the
result in some time :wink:

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.