[Geoserver-devel] App-schema mapping creation/update via REST

GeoServer GeoServer Developer
,
1

Dear all,
I’m working on automating the creation/update of app-schema datastores in GeoServer leveraging the REST API.
The current API already allows the creation of an app-schema datastore and, of course, it is already possible to create secondary namespaces/workspaces via REST: what is missing (at least AFAIK) is the possibility to upload an app-schema mapping file to an already created app-schema datastore (also creating a new one based on the uploaded mapping file would be nice).

I had a quick look at the REST endpoints we already have and I focused on this one:
/workspaces//datastores//[file|url|external][.extension]

which is ultimately handled by either DataStoreFileResource or CoverageStoreFileResource. The supported extensions so far are shape, properties, h2 and spatialite.

I’m proposing to add a new extension, appschema, that will be handled by a new kind of resource, let’s call it AppSchemaDataAccessResource, extending StoreFileResource, which will take care of copying the uploaded mapping file to its proper location.
I’m a bit puzzled as to where (i.e. in which module) this new class should be put: I guess gs-restconfig would be the wrong place, since it would make it dependent on gt-app-schema. Perhaps the best way would be to create a new module, let’s call it gt-app-schema-rest, containing just AppSchemaDataAccessResource and occasional supporting classes, and then have StoreFileFinder (gs-restconfig module) somehow look it up dynamically.

Any thoughts/suggestions?

Thanks!

Best regards,
Stefano Costa

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Dott. Stefano Costa
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate.
Il loro utilizzo è consentito esclusivamente al destinatario del
messaggio, per le finalità indicate nel messaggio stesso. Qualora
riceviate questo messaggio senza esserne il destinatario, Vi preghiamo
cortesemente di darcene notizia via e-mail e di procedere alla
distruzione del messaggio stesso, cancellandolo dal Vostro sistema.
Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità
diverse, costituisce comportamento contrario ai principi dettati dal
D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely
for the attention and use of the named addressee(s) and may be
confidential or proprietary in nature or covered by the provisions of
privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New
Data Protection Code).Any use not in accord with its purpose, any
disclosure, reproduction, copying, distribution, or either
dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the
intended recipient, please contact immediately the sender by
telephone, fax or e-mail and delete the information in this message
that has been received in error. The sender does not give any warranty
or accept liability as the content, accuracy or completeness of sent
messages and accepts no responsibility for changes made after they
were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.

2

On Wed, Jun 10, 2015 at 5:53 PM, stefano.costa <
stefano.costa@anonymised.com> wrote:

Dear all,
I'm working on automating the creation/update of app-schema datastores in
GeoServer leveraging the REST API.
The current API already allows the creation of an app-schema datastore
and, of course, it is already possible to create secondary
namespaces/workspaces via REST: what is missing (at least AFAIK) is the
possibility to upload an app-schema mapping file to an already created
app-schema datastore (also creating a new one based on the uploaded mapping
file would be nice).

Hi Stefano,
trying to answer, but I'm not 100% familar with that part of REST, and it's
late here, so I apologize if I'm off the mark

In the REST api those are two different use cases. In vector stores, you
PUT a shp/property/h2/spatialite to create a new store,
and the file location you provides is filled as part of the store creation
parameter maps.
If there store is there and you are created a new feature type, you instead
POST to that feature type with a schema definition,
and the code will call createSchema().

From the docs App-schema seems to be taking the url to a configuration

file, if so, I guess you could just modify the
existing DataStoreFileResource to handle that case (like, spatialite is
not a core module anyways, but since
everything is working via strings and maps, you don't really need to add
dependencies).
I'm not totally clear with the details though, like I guess you also need
to include the schema and so on in the
zip file you're sending or not?

Adding a mapping to an existing store instead, uh, not sure what to answer
to that one, it does not look that app-schema
has any facility to add feature types at runtime, looks like you have to
update the main config file and rebuilt the
app-schema store...
Logically it would seem the code handling this would have to go around this
area:
https://github.com/geoserver/geoserver/blob/master/src/restconfig/src/main/java/org/geoserver/catalog/rest/FeatureTypeResource.java#L86

Or else, maybe you can register a new resource under:

/workspaces/{workspace}/datastores/{datastore}/featuretypes/{featuretype}/mapping

That would be a special handler for just the app-schema mappings, a GET
would return the mapping, a PUT would
create it, and as a consequence, create the FeatureTypeInfo as well?

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

3

This is kind of tricky as general problem - how do we manage configuration files such as required for templates, icons, app-schema mapping configuration, pregeneralized datastore config etc…

I think we have now isolated all file access to ResourceStore API - and I had hoped to see a simple REST API added to manage configuration objects such as this.

···

On 10 June 2015 at 08:53, stefano.costa <stefano.costa@anonymised.com> wrote:

Dear all,
I’m working on automating the creation/update of app-schema datastores in GeoServer leveraging the REST API.
The current API already allows the creation of an app-schema datastore and, of course, it is already possible to create secondary namespaces/workspaces via REST: what is missing (at least AFAIK) is the possibility to upload an app-schema mapping file to an already created app-schema datastore (also creating a new one based on the uploaded mapping file would be nice).

I had a quick look at the REST endpoints we already have and I focused on this one:
/workspaces//datastores//[file|url|external][.extension]

which is ultimately handled by either DataStoreFileResource or CoverageStoreFileResource. The supported extensions so far are shape, properties, h2 and spatialite.

I’m proposing to add a new extension, appschema, that will be handled by a new kind of resource, let’s call it AppSchemaDataAccessResource, extending StoreFileResource, which will take care of copying the uploaded mapping file to its proper location.
I’m a bit puzzled as to where (i.e. in which module) this new class should be put: I guess gs-restconfig would be the wrong place, since it would make it dependent on gt-app-schema. Perhaps the best way would be to create a new module, let’s call it gt-app-schema-rest, containing just AppSchemaDataAccessResource and occasional supporting classes, and then have StoreFileFinder (gs-restconfig module) somehow look it up dynamically.

Any thoughts/suggestions?

Thanks!

Best regards,
Stefano Costa

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Dott. Stefano Costa
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate.
Il loro utilizzo è consentito esclusivamente al destinatario del
messaggio, per le finalità indicate nel messaggio stesso. Qualora
riceviate questo messaggio senza esserne il destinatario, Vi preghiamo
cortesemente di darcene notizia via e-mail e di procedere alla
distruzione del messaggio stesso, cancellandolo dal Vostro sistema.
Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità
diverse, costituisce comportamento contrario ai principi dettati dal
D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely
for the attention and use of the named addressee(s) and may be
confidential or proprietary in nature or covered by the provisions of
privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New
Data Protection Code).Any use not in accord with its purpose, any
disclosure, reproduction, copying, distribution, or either
dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the
intended recipient, please contact immediately the sender by
telephone, fax or e-mail and delete the information in this message
that has been received in error. The sender does not give any warranty
or accept liability as the content, accuracy or completeness of sent
messages and accepts no responsibility for changes made after they
were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Jody Garnett

4

Hi Andrea,
thanks for your answer, see my comments inline.

Il giorno mer, 10/06/2015 alle 20.31 +0200, Andrea Aime ha scritto:

In the REST api those are two different use cases. In vector stores,
you PUT a shp/property/h2/spatialite to create a new store,
and the file location you provides is filled as part of the store
creation parameter maps.
If there store is there and you are created a new feature type, you
instead POST to that feature type with a schema definition,
and the code will call createSchema().

From the docs App-schema seems to be taking the url to a configuration
file, if so, I guess you could just modify the
existing DataStoreFileResource to handle that case (like, spatialite
is not a core module anyways, but since
everything is working via strings and maps, you don't really need to
add dependencies).

I tried to modify DataStoreFileResource as you suggested and the first
hurdle I bumped in is that, as the name suggests, the implementation is
based off DataStoreFactorySpi/DataStore, while app-schema uses directly
DataAccessFactory/DataAccess. I tried to replace all instances of
DataStoreFactorySpi/DataStore with DataAccessFactory/DataAccess and it
didn't turn out so bad... not sure though if this is the best way to do
it; another possibility that comes to my mind would be to create a
DataAccessFileResource, that DataStoreFileResource would then extend.

I'm not totally clear with the details though, like I guess you also
need to include the schema and so on in the
zip file you're sending or not?

AFAIK, it's not strictly necessary: if the mapping file contains a URL
pointing to a remote schema, the app-schema plugin will download it and
put it in the app-schema cache at first access.

Adding a mapping to an existing store instead, uh, not sure what to
answer to that one, it does not look that app-schema
has any facility to add feature types at runtime, looks like you have
to update the main config file and rebuilt the
app-schema store...

Yeah, that's exactly what I had in mind, at least for now: no
possibility to add new feature types at runtime, just allow for a full
rewrite of the mapping file the store is based off, which implies
replacing the existing feature types with the ones described in the new
mapping file.
If replacing the mapping file, with all that this implies, doesn't work
out, clients could just resort to dropping the datastore and creating it
anew.

Logically it would seem the code handling this would have to go around
this area:
https://github.com/geoserver/geoserver/blob/master/src/restconfig/src/main/java/org/geoserver/catalog/rest/FeatureTypeResource.java#L86

Or else, maybe you can register a new resource under:

/workspaces/{workspace}/datastores/{datastore}/featuretypes/{featuretype}/mapping

That would be a special handler for just the app-schema mappings, a
GET would return the mapping, a PUT would
create it, and as a consequence, create the FeatureTypeInfo as well?

Sounds like a good idea to me, but, as I wrote above, I'd be content
with much less for now :slight_smile:

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

--

Best regards,
Stefano Costa

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Dott. Stefano Costa
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate.
Il loro utilizzo è consentito esclusivamente al destinatario del
messaggio, per le finalità indicate nel messaggio stesso. Qualora
riceviate questo messaggio senza esserne il destinatario, Vi preghiamo
cortesemente di darcene notizia via e-mail e di procedere alla
distruzione del messaggio stesso, cancellandolo dal Vostro sistema.
Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità
diverse, costituisce comportamento contrario ai principi dettati dal
D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely
for the attention and use of the named addressee(s) and may be
confidential or proprietary in nature or covered by the provisions of
privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New
Data Protection Code).Any use not in accord with its purpose, any
disclosure, reproduction, copying, distribution, or either
dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the
intended recipient, please contact immediately the sender by
telephone, fax or e-mail and delete the information in this message
that has been received in error. The sender does not give any warranty
or accept liability as the content, accuracy or completeness of sent
messages and accepts no responsibility for changes made after they
were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.

5

On Wed, Jun 10, 2015 at 10:16 PM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

This is kind of tricky as general problem - how do we manage configuration
files such as required for templates, icons, app-schema mapping
configuration, pregeneralized datastore config etc...

I think we have now isolated all file access to ResourceStore API
<https://github.com/geoserver/geoserver/wiki/ResourceStore-API-Examples&gt; -
and I had hoped to see a simple REST API added to manage configuration
objects such as this.

I am not sure the ResoruceStore API is suitable for this task, the
configuration file can be anywhere on the file system, does not
necessarily have to be in the data directory?

When uploading new file we have to respect the decisions of the path
mapper, while the default one works inside
the data directory, the general implementation of it it's not limited to
it, and indeed, it's normally not adding data to the
data dir contents (for good reasons, in a production setting you don't want
to mix configuration and data in the
same directory tree):

https://github.com/geoserver/geoserver/blob/master/src/rest/src/main/java/org/geoserver/rest/util/RESTUploadPathMapper.java

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

6

On Thu, Jun 11, 2015 at 1:04 PM, Stefano Costa <
stefano.costa@anonymised.com> wrote:

I tried to modify DataStoreFileResource as you suggested and the first
hurdle I bumped in is that, as the name suggests, the implementation is
based off DataStoreFactorySpi/DataStore, while app-schema uses directly
DataAccessFactory/DataAccess. I tried to replace all instances of
DataStoreFactorySpi/DataStore with DataAccessFactory/DataAccess and it
didn't turn out so bad... not sure though if this is the best way to do
it; another possibility that comes to my mind would be to create a
DataAccessFileResource, that DataStoreFileResource would then extend.

I guess the latter is better, or at least, it sounds more in line with how
the
difference between store and data access is often managed, but if you
already did the former, that works too, could you share a link to
a github diff for it?

> I'm not totally clear with the details though, like I guess you also
> need to include the schema and so on in the
> zip file you're sending or not?

AFAIK, it's not strictly necessary: if the mapping file contains a URL
pointing to a remote schema, the app-schema plugin will download it and
put it in the app-schema cache at first access.

Nice.

>
> Adding a mapping to an existing store instead, uh, not sure what to
> answer to that one, it does not look that app-schema
> has any facility to add feature types at runtime, looks like you have
> to update the main config file and rebuilt the
> app-schema store...

Yeah, that's exactly what I had in mind, at least for now: no
possibility to add new feature types at runtime, just allow for a full
rewrite of the mapping file the store is based off, which implies
replacing the existing feature types with the ones described in the new
mapping file.
If replacing the mapping file, with all that this implies, doesn't work
out, clients could just resort to dropping the datastore and creating it
anew.

Ok. So... normally you would use the POST operation to create a new store,
but a PUT operation to update an existing one.
I believe that rigth now we have PUT in terms of updating the StoreInfo
xml config, but not a version of it that would replace the "source data"
(where in app-schema,
the closest equivalent seems to be indeed the mapping file, as everything
else
is already registered as its own store in GeoServer).

Also, what happens if you are trying to update a config file for a
pre-existing
app-schema store that was hand-crafted (and which might have a different
path to the config one, than the one the rest path mappers would like to use
for your store?).

In general, it may seem you'd need to roll some custom behavior to locate
the right mapping file based on the existing configuration, which would make
for app-schema specific code.

How are you going to handle it?

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

7

Hi Andrea,

Il giorno ven, 12/06/2015 alle 05.20 +0200, Andrea Aime ha scritto:

On Thu, Jun 11, 2015 at 1:04 PM, Stefano Costa <stefano.costa@anonymised.com> wrote:

I guess the latter is better, or at least, it sounds more in line with how the

difference between store and data access is often managed, but if you

already did the former, that works too, could you share a link to

a github diff for it?

As a matter of fact, I do have a draft implementation of the former: https://github.com/geoserver/geoserver/compare/master…ridethepenguin:app-schema-rest
I’ll frankly admit the code is not pretty, so I’m very much open to suggestions on how to improve it :slight_smile:

Ok. So… normally you would use the POST operation to create a new store,

but a PUT operation to update an existing one.

I believe that rigth now we have PUT in terms of updating the StoreInfo

xml config, but not a version of it that would replace the “source data” (where in app-schema,

the closest equivalent seems to be indeed the mapping file, as everything else

is already registered as its own store in GeoServer).

As per the documentation, a PUT request to the endpoint:
/workspaces/<ws>/datastores/<ds>/[file|url|external][.<extension>]

will “upload files to the datastore (in our case, the mapping file), creating it if necessary”.

For example, I successfully created a new app-schema datastore issuing this PUT request:
curl -X PUT -d @LandCoverVector.xml -H "Content-Type: text/xml" -u admin:geoserver http://localhost:8080/geoserver/rest/workspaces/lcv/datastores/LandCoverVector/file.appschema?configure=all

where LandCoverVector.xml is an app-schema mapping file containing the mappings for two feature types, LandCoverDataset and LandCoverUnit. The datastore was successfully created and the feature types in it correctly configured, thanks to the configure=all request parameter.

Also, what happens if you are trying to update a config file for a pre-existing

app-schema store that was hand-crafted (and which might have a different

path to the config one, than the one the rest path mappers would like to use

for your store?).

In general, it may seem you’d need to roll some custom behavior to locate

the right mapping file based on the existing configuration, which would make

for app-schema specific code.

How are you going to handle it?

I tested this use case and (I believe) the service did the right thing, without app-schema specific code, updating the url in the datastore.xml file to point to the location of the uploaded mapping file. The only glitch I saw here was that the mapping file was stored outside the datastore directory $GEOSERVER_DATA_DIR/workspaces//, under $GEOSERVER_DATA_DIR/data/…

My hand-crafted datastore.xml file went like this:
…
<entry key="url">file:workspaces/lcv/LandCoverVector/LandCoverVector.xml</entry>
…

I issued the request:
curl -X PUT -d @LandCoverVector_alternative.xml -H "Content-Type: text/xml" -u admin:geoserver [http://localhost:8080/geoserver/rest/workspaces/lcv/datastores/LandCoverVector/file.appschema?configure=none](http://localhost:8080/geoserver/rest/workspaces/lcv/datastores/LandCoverVector/file.appschema?configure=none)

where LandCoverVector_alternative.xml is a mapping file with slightly different mappings for the same feature types (no need to configure them again, so configure=none)

As a result, the datastore.xml was updated as such:
<entry key="url">file:/home/stefano/Lavoro/GeoSolutions/projects/hale/geoserver-2.6.2/data_dir/data/lcv/LandCoverVector/LandCoverVector.appschema</entry>

I issued a couple of WFS requests to confirm that the mappings had been updated, and in fact they were.

Now, this worked out nicely because the feature types in the two mapping files were exactly the same: in case the new mapping file contains new feature types or omits some that were present in the old mapping, perhaps the best way to update the configuration would be to DELETE the entire store and create it again issuing a PUT request with the new mapping file in the body and the configure paramter set to all.

What do you think?

Cheers

Andrea



<br>-- <br><br>Best regards,<br>Stefano Costa<br><br>==<br>GeoServer Professional Services from the experts! Visit<br>http://goo.gl/it488V for more information.<br>==<br>Dott. Stefano Costa<br>Senior Software Engineer<br><br>GeoSolutions S.A.S.<br>Via Poggio alle Viti 1187<br>55054 Massarosa (LU)<br>Italy<br>phone: +39 0584 962313<br>fax: +39 0584 1660272<br><br>http://www.geo-solutions.it<br>http://twitter.com/geosolutions_it<br><br>-------------------------------------------------------<br>AVVERTENZE AI SENSI DEL D.Lgs. 196/2003<br>Le informazioni contenute in questo messaggio di posta elettronica e/o<br>nel/i file/s allegato/i sono da considerarsi strettamente riservate.<br>Il loro utilizzo è consentito esclusivamente al destinatario del<br>messaggio, per le finalità indicate nel messaggio stesso. Qualora<br>riceviate questo messaggio senza esserne il destinatario, Vi preghiamo<br>cortesemente di darcene notizia via e-mail e di procedere alla<br>distruzione del messaggio stesso, cancellandolo dal Vostro sistema.<br>Conservare il messaggio stesso, divulgarlo anche in parte,<br>distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità<br>diverse, costituisce comportamento contrario ai principi dettati dal<br>D.Lgs. 196/2003.<br><br>The information in this message and/or attachments, is intended solely<br>for the attention and use of the named addressee(s) and may be<br>confidential or proprietary in nature or covered by the provisions of<br>privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New<br>Data Protection Code).Any use not in accord with its purpose, any<br>disclosure, reproduction, copying, distribution, or either<br>dissemination, either whole or partial, is strictly forbidden except<br>previous formal approval of the named addressee(s). If you are not the<br>intended recipient, please contact immediately the sender by<br>telephone, fax or e-mail and delete the information in this message<br>that has been received in error. The sender does not give any warranty<br>or accept liability as the content, accuracy or completeness of sent<br>messages and accepts no responsibility for changes made after they<br>were sent or for other risks which arise as a result of e-mail<br>transmission, viruses, etc.<br><br><br><br>
8

On Fri, Jun 12, 2015 at 11:55 AM, Stefano Costa <
stefano.costa@anonymised.com> wrote:

As a matter of fact, I do have a draft implementation of the former:
https://github.com/geoserver/geoserver/compare/master...ridethepenguin:app-schema-rest

Hi Stefano,
maybe next time you want to discuss some change make a pull request, we
have better tools
to comment and discuss if you do.

Anyways, here is a quick review:
1) Avoid long winded comments about how the code used to look compared to
how it looks like
    in
https://github.com/geoserver/geoserver/compare/master...ridethepenguin:app-schema-rest#diff-1c2ca55edfdb0e4afc6249958dbe0bacR316
    just state why you are doing it, e.g. "this code makes sure we do not
create two app-schema stores with the same url, as that
    would result in an exception"
2) These checks should be un-necessary, the store just gave you the names,
they should be usable without extracting the local
     part:
https://github.com/geoserver/geoserver/compare/master...ridethepenguin:app-schema-rest#diff-1c2ca55edfdb0e4afc6249958dbe0bacR416
     If really needed, we might have a bug or other type of problem, let's
discuss
3) I'd change the "disposableSource" into "createNewSource", all DataAccess
have a dispose method, they are all disposable,
     we have to call dispose only if we created a new store/dataaccess via
its factory, instead of having it cached by resource pool

For the rest no particular comment (besides that the original code was
already scary), but we need a test please :slight_smile:
(adding a test scoped dependency to app-schema just to run the test)

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

9

Hi Andrea,
thanks for the review, I intended to issue a PR earlier but couldn't
make it in time. Here it is:
https://github.com/geoserver/geoserver/pull/1106

Unfortunately, I noticed your e-mail only after creating the PR, so the
code does not take into account your comments. Tomorrow I'll update it
accordingly.

Il giorno mar, 16/06/2015 alle 18.35 +0200, Andrea Aime ha scritto:

On Fri, Jun 12, 2015 at 11:55 AM, Stefano Costa
<stefano.costa@anonymised.com> wrote:
        > As a matter of fact, I do have a draft implementation of the
        > former:
        > https://github.com/geoserver/geoserver/compare/master...ridethepenguin:app-schema-rest

Hi Stefano,
maybe next time you want to discuss some change make a pull request,
we have better tools
to comment and discuss if you do.

Anyways, here is a quick review:
1) Avoid long winded comments about how the code used to look compared
to how it looks like

in https://github.com/geoserver/geoserver/compare/master...ridethepenguin:app-schema-rest#diff-1c2ca55edfdb0e4afc6249958dbe0bacR316
    just state why you are doing it, e.g. "this code makes sure we do
not create two app-schema stores with the same url, as that
    would result in an exception"
2) These checks should be un-necessary, the store just gave you the
names, they should be usable without extracting the local

part: https://github.com/geoserver/geoserver/compare/master...ridethepenguin:app-schema-rest#diff-1c2ca55edfdb0e4afc6249958dbe0bacR416
     If really needed, we might have a bug or other type of problem,
let's discuss
3) I'd change the "disposableSource" into "createNewSource", all
DataAccess have a dispose method, they are all disposable,
     we have to call dispose only if we created a new store/dataaccess
via its factory, instead of having it cached by resource pool

For the rest no particular comment (besides that the original code was
already scary), but we need a test please :slight_smile:
(adding a test scoped dependency to app-schema just to run the test)

Cheers
Andrea

--

Best regards,
Stefano Costa

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Dott. Stefano Costa
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate.
Il loro utilizzo è consentito esclusivamente al destinatario del
messaggio, per le finalità indicate nel messaggio stesso. Qualora
riceviate questo messaggio senza esserne il destinatario, Vi preghiamo
cortesemente di darcene notizia via e-mail e di procedere alla
distruzione del messaggio stesso, cancellandolo dal Vostro sistema.
Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità
diverse, costituisce comportamento contrario ai principi dettati dal
D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely
for the attention and use of the named addressee(s) and may be
confidential or proprietary in nature or covered by the provisions of
privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New
Data Protection Code).Any use not in accord with its purpose, any
disclosure, reproduction, copying, distribution, or either
dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the
intended recipient, please contact immediately the sender by
telephone, fax or e-mail and delete the information in this message
that has been received in error. The sender does not give any warranty
or accept liability as the content, accuracy or completeness of sent
messages and accepts no responsibility for changes made after they
were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.

10

Hi Andrea,
I updated the PR according to your suggestions:
https://github.com/geoserver/geoserver/pull/1106

Il giorno mar, 16/06/2015 alle 18.35 +0200, Andrea Aime ha scritto:

2) These checks should be un-necessary, the store just gave you the
names, they should be usable without extracting the local

I introduced them because namespace was not added to the parameters
passed to factory.createDataStore(); now I added it and removed the
check.

For the rest no particular comment (besides that the original code was
already scary), but we need a test please :slight_smile:
(adding a test scoped dependency to app-schema just to run the test)

Added a test scoped dependency to gt-app-schema in gs-restconfig's POM
file. Unit tests are all green for me :slight_smile:

Cheers
Andrea

--

Best regards,
Stefano Costa

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Dott. Stefano Costa
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate.
Il loro utilizzo è consentito esclusivamente al destinatario del
messaggio, per le finalità indicate nel messaggio stesso. Qualora
riceviate questo messaggio senza esserne il destinatario, Vi preghiamo
cortesemente di darcene notizia via e-mail e di procedere alla
distruzione del messaggio stesso, cancellandolo dal Vostro sistema.
Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità
diverse, costituisce comportamento contrario ai principi dettati dal
D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely
for the attention and use of the named addressee(s) and may be
confidential or proprietary in nature or covered by the provisions of
privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New
Data Protection Code).Any use not in accord with its purpose, any
disclosure, reproduction, copying, distribution, or either
dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the
intended recipient, please contact immediately the sender by
telephone, fax or e-mail and delete the information in this message
that has been received in error. The sender does not give any warranty
or accept liability as the content, accuracy or completeness of sent
messages and accepts no responsibility for changes made after they
were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.