[Geoserver-users] Create Directory of Shape files Datastore with REST API & JSON

GeoServer GeoServer User
1

Hello all,

Please could I ask for a little help – I am trying to create a datastore from a dir of shapefiles already on the server, using JSON via REST API.

I am unsure of the correct data structure to post to the server, as the documentation is not quite exhaustive in this area.

At the moment I am trying the following:

POST to httrp://server:port/rest/workspaces/myWorkspace/datastores

Body:

dataStore: {

name: ‘name’,

description: ‘my description’,

enabled: true,

connectionParameters: {

url: “file:data/dirName”,

fstype: “shape”

}

}

Any help as always, appreciated.

Mark

2

On Fri, Jun 6, 2014 at 12:45 PM, Mark Paxton <mark.paxton@anonymised.com>
wrote:

Hello all,

Please could I ask for a little help – I am trying to create a datastore
from a dir of shapefiles already on the server, using JSON via REST API.

I am unsure of the correct data structure to post to the server, as the
documentation is not quite exhaustive in this area.

Whenever something is not documented there is an easy way to discover the
data structure you need:
* go into the user interface, create the objects you need
* go into the rest api, and grab their xml/json representation

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

3

Andrea,

Thanks for the reply, you are usually very helpful with my queries to the geoserver lists, but really that is not a very constructive reply today.

I have already tried many things before posting to the list and I looked at the REST API in detail. It is not clear exactly how the description of the datastore maps to the information to create it. Perhaps you have not looked either, as existing shape file directories are described as below, but that leaves much to be desired in understanding how to create. Please see the comments after.

{

“dataStore”:{

“name”:“myStore”,

“type”:“Directory of spatial files (shapefiles)”,

“enabled”:true,

“workspace”:{

“name”:“myWs”,

“href”:“http://gis.mysite.com/rest/workspaces/myws.json”

},

“connectionParameters”:{

“entry”:[

{“@key”:“memory mapped buffer”,“$”:“false”},

{“@key”:“timezone”,“$”:“Europe/London”},

{“@key”:“fstype”,“$”:“shape”},

{“@key”:“create spatial index”,“$”:“true”},

{“@key”:“charset”,“$”:“ISO-8859-1”},

{“@key”:“filetype”,“$”:“shapefile”},

{“@key”:“cache and reuse memory maps”,“$”:“false”},

{“@key”:“enable spatial index”,“$”:“true”},

{“@key”:“url”,“$”:“file:gisdata/myData”},

{“@key”:“namespace”,“$”:“wheatstone”}

]

},

“_default”:false,

“featureTypes”:“http://gis.seaplanner.com/rest/workspaces/myws/datastores/mystore/featuretypes.json”

}

}

For a start, name and workspace are specified in the POST url, so I guess those are optional?

The href attributes as well, seem unnecessary for a create, so I guess these should not be needed?

Posting type as “Directory of spatial files (shapefiles)” would also seem a bit odd, is that really the way?

As far as connection parameters, are they really specified in this convoluted format? And will it use defaults not set?

Same with “_default” why underscore? And is it Optional, default false?

featureTypes again, this is like the href attributes, seems inappropriate for a create/post, presumably excluded.

So really, what I am still asking is as before, what should be the right data structure?

The curl example, only seems to describe creation with name and workspace, putting the file path in the body:

··· 
curl -v -u admin:geoserver -XPUT -H "Content-type: text/plain"

  -d "file:///data/shapefiles/"

  "http://localhost:8080/geoserver/rest/workspaces/acme/datastores/shapefiles/external.shp?configure=all"

If I want a name, description it is not clear.

Mark

From: andrea.aime@…84… [mailto:andrea.aime@…84…] On Behalf Of Andrea Aime
Sent: 06 June 2014 11:55
To: Mark Paxton
Cc: Geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] Create Directory of Shape files Datastore with REST API & JSON

On Fri, Jun 6, 2014 at 12:45 PM, Mark Paxton <mark.paxton@…6564…> wrote:

Hello all,

Please could I ask for a little help – I am trying to create a datastore from a dir of shapefiles already on the server, using JSON via REST API.

I am unsure of the correct data structure to post to the server, as the documentation is not quite exhaustive in this area.

Whenever something is not documented there is an easy way to discover the data structure you need:

  • go into the user interface, create the objects you need

  • go into the rest api, and grab their xml/json representation

Cheers

Andrea

–

==

GeoServer Professional Services from the experts! Visit

http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf

Technical Lead

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

4

On Fri, Jun 6, 2014 at 2:30 PM, Mark Paxton <mark.paxton@anonymised.com>
wrote:

Andrea,

Thanks for the reply, you are usually very helpful with my queries to the
geoserver lists, but really that is not a very constructive reply today.

Sorry, it's all the reply I could afford... I don't know what the structure
would be myself, and I suggested the way I would move if I was the one that
had to configure a shapefile of directories store.
If that's not good enough, you have two options:
* look into the code and figure out how things work (that's normally my
plan B)
* get someone to do it for you and write better documentation as a result.

Given I have very little experience with the REST api code and I don't know
all the answers you need by heart, I'll let someone
that does (hopefully) follow up

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

5

As you suggest, I was already checking the source code for this.

From what I can tell, there’s no direct way to create a datasource like this from the body of a JSON request.

Perhaps I am wrong, but it will take the author of the code to tell us how as it is quite a convoluted process to understand.

For now the simplest way is make a PUT request as per:

PUT: http://localhost:8080/rest/workspaces/WORKSPACENAME/datastores/DATASTORENAME/external.shp

In this url, you have 2 parameters:

First:

external: refers to a file external to geoserver on your sever.

url: Get the content from the url specified in the body of the put.

file: directly upload a file in the body of your put.

Second:

shp: Shapefile format

properties: java properties file

h2: h2 database

spatialite: spatialite database

The body of your put will either be a string to the location of your files, e.g.

file://var/lib/gisdata/ (this refers to a directory, though you can point to a single shape file)

or

http://data.geoserver.com/shapes.shp

Alternatively if you use the file option, you put your file in the post and it’s saved on the sever.

Hopefully that will help a few people!

Interestingly, there’s a start at a GeoServer javascript client API in github based on openlayers:

https://github.com/geoserver/geoserver/tree/master/src/restconfig/src/client/javascript

Checked in by aaime on 18 Jun 2011…

Regards,

Mark

···

Thanks for the reply, you are usually very helpful with my queries to the geoserver lists, but really that is not a very constructive reply today.

Sorry, it’s all the reply I could afford… I don’t know what the structure would be myself, and I suggested the way I would move if I was the one that had to configure a shapefile of directories store.

If that’s not good enough, you have two options:

  • look into the code and figure out how things work (that’s normally my plan B)

  • get someone to do it for you and write better documentation as a result.

Given I have very little experience with the REST api code and I don’t know all the answers you need by heart, I’ll let someone

that does (hopefully) follow up

Cheers

Andrea

–

==

GeoServer Professional Services from the experts! Visit

http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf

Technical Lead

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

6

On Fri, Jun 6, 2014 at 6:01 PM, Mark Paxton <mark.paxton@anonymised.com>
wrote:

Interestingly, there’s a start at a GeoServer javascript client API in
github based on openlayers:

https://github.com/geoserver/geoserver/tree/master/src/restconfig/src/client/javascript

Checked in by aaime on 18 Jun 2011…

Nah, that's not me, I don't do javascript. It's just that the history of
the normal
repo got cut for performance reasons (we had many large binary checkins back
when we were using svn).

The full history of the old repo is available here:
https://github.com/geoserver/geoserver-history

As you can see from the history, it was checked in by the original author
of the rest-config
module, Justin Deoliveira:
https://github.com/geoserver/geoserver-history/commits/master/src/restconfig/src/client/javascript/lib

Cheers
Andrea

--

Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

7

I've been dealing with this situation for a couple of days now and here's
what I found

First, your original question includes an example that WORKS if you POST it
to

/rest/workspaces/topp/datastores.json

with a header setting "Content-type: application/json". It will create a
datastore of the type "Shapefile" even if your "url" parameters point to a
directory that holds a shapefile. (/I thought I should mention this because
the web interface only lets you create a "Shapefile" datastore if you point
'url' to an specific shapefile and won let you pick just a directory/).

Your example doesn't work if you POST it to
/rest/workspaces/myWorkspace/datastores/name because that's not the REST
endpoint for creating datastores.

This is the minimum object you should POST if you specifically want to
create a data store of the type "Directory of spatial files (shapefiles)"
instead of "Shapefile" datastore.

"dataStore": {
      "name": "name",
      "type": "Directory of spatial files (shapefiles)",
      "connectionParameters": {
        "url": "file:data/shapefiles",
        "fstype": "shape"
      }
}

What Andreas said is true.

geowolf wrote

Whenever something is not documented there is an easy way to discover the
data structure you need:
* go into the user interface, create the objects you need
* go into the rest api, and grab their xml/json representation

The problem with this being seamlessly useful is a REST API implementation
detail that makes it represent some plain objects in a strange way when you
ask the API for a JSON representation about any resource. I mean, when the
REST API tells you something like

"connectionParameters":{
  "entry":[
    {"@key":"memory mapped buffer","$":"false"},
    {"@key":"fstype","$":"shape"}
  ]
}

it means that what you have to POST for a new datastore is:

"connectionParameters":{
    "memory mapped buffer":"false",
    "fstype":"shape"
}

I think this is due to Geoserver or the REST module using a specific library
to convert to JSON objects that behaves in that way when serializing a JAVA
object

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Create-Directory-of-Shape-files-Datastore-with-REST-API-JSON-tp5144395p5147634.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

8

I've been dealing with this situation for a couple of days now and here's
what I found

First, your original question includes an example that WORKS if you POST it
to

/rest/workspaces/topp/datastores.json

with a header setting "Content-type: application/json". It will create a
datastore of the type "Shapefile" even if your "url" parameters point to a
directory that holds a shapefile. (/I thought I should mention this because
the web interface only lets you create a "Shapefile" datastore if you point
'url' to an specific shapefile and won let you pick just a directory/).

Your example doesn't work if you POST it to
/rest/workspaces/myWorkspace/datastores/name because that's not the REST
endpoint for creating datastores.

This is the minimum object you should POST if you specifically want to
create a data store of the type "Directory of spatial files (shapefiles)"
instead of "Shapefile" datastore.

"dataStore": {
      "name": "name",
      "type": "Directory of spatial files (shapefiles)",
      "connectionParameters": {
        "url": "file:data/shapefiles",
        "fstype": "shape"
      }
}

What Andreas said is true.

geowolf wrote

Whenever something is not documented there is an easy way to discover the
data structure you need:
* go into the user interface, create the objects you need
* go into the rest api, and grab their xml/json representation

The problem with this being seamlessly useful is a REST API implementation
detail that makes it represent some plain objects in a strange way when you
ask the API for a JSON representation about any resource. I mean, when the
REST API tells you something like

"connectionParameters":{
  "entry":[
    {"@key":"memory mapped buffer","$":"false"},
    {"@key":"fstype","$":"shape"}
  ]
}

it means that what you have to POST for a new datastore is:

"connectionParameters":{
    "memory mapped buffer":"false",
    "fstype":"shape"
}

I think this is due to Geoserver or the REST module using a specific library
to convert to JSON objects that behaves in that way when serializing a JAVA
object

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Create-Directory-of-Shape-files-Datastore-with-REST-API-JSON-tp5144395p5147893.html
Sent from the GeoServer - User mailing list archive at Nabble.com.