[Geoserver-devel] Exposing the ability to reload a feature type in the REST API

Hi,
currently if someone needs to reload a specific feature type definition/structure they can only do so from the UI,
but not from the REST API. A use case here is something changing the structure of the feature type in the storage,
and the consequent need to inform GeoServer about it.

The REST API only offers a full resource pool reset, and even with the “reload feature type” button in the UI, we
are looking at a somewhat blunt instrument, in that it’s actually removing from the resource pool (and thus disposing)
the entire store attached to it:

https://github.com/geoserver/geoserver/blob/master/src/web/core/src/main/java/org/geoserver/web/data/resource/FeatureResourceConfigurationPanel.java#L132

This in turn might affect all in flight operations attached to the store.

How about we change that a little, so that:

  • The store cleaning, if necessary, is done inside ResourcePool.clear(featureType)
  • The cleaning is done in an intelligent way, and if the store is a ContentDataStore, we only call store.getEntry(name).dispose()?
    (btw, can anyone confirm that is not going to cause issues? any better way to clear the feature type to force the store to re-compute it?)
    And then, have FeatureTypeResource also leverage that code.

In terms of REST API change, I was thinking to mimic the existing reset endpoint, and perform resetting on a feature type if a empty POST is done against the resource, along with a ?reset=true parameter (missing a more significant HTTP method to do so), e.g.:

http://host/geoserver/rest/workspaces/topp/datastores/states_shapefile/featuretypes/states?reset=true

A possible alternative would be to build inside the existing reset path instead, something like a empty POST to:

http://host/geoserver/rest/reset/workspaces/topp/datastores/states_shapefile/featuretypes/states

Opinions?

Cheers

Andrea

···

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On 04/11/16 23:14, Andrea Aime wrote:

And then, have FeatureTypeResource also leverage that code.
In terms of REST API change, I was thinking to mimic the existing reset
endpoint, and perform resetting on a feature type if a empty POST is done
against the resource, along with a ?reset=true parameter (missing a more
significant HTTP method to do so), e.g.:
http://host/geoserver/rest/workspaces/topp/datastores/states_shapefile/featuretypes/states?reset=true

This seems the more elegant option to me. Perhaps "?reload=true" as reset might imply undoing some change? You wrote "reload" in the title and that seemed clear to me.

I wonder if this new functionality has any implications for app-schema with its cooperating data stores and static data store registry?

Kind regards,

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <http://transient.nz/&gt;
New Zealand

I had a JIRA issue on this a while back; when we were looking at how to let GeoServer know what to do when a new column was added to a postgis table.
We ended up with a recommendation for something similar to recalculate, but with different options, including one that would regenerate the cache of attributes like the GUI (but funding never came through).

There is a better way, ContentDataStore has an method to clear is ContentEntry (which caches the schema at the DataStore level).

···

On 4 November 2016 at 03:14, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
currently if someone needs to reload a specific feature type definition/structure they can only do so from the UI,
but not from the REST API. A use case here is something changing the structure of the feature type in the storage,
and the consequent need to inform GeoServer about it.

The REST API only offers a full resource pool reset, and even with the “reload feature type” button in the UI, we
are looking at a somewhat blunt instrument, in that it’s actually removing from the resource pool (and thus disposing)
the entire store attached to it:

https://github.com/geoserver/geoserver/blob/master/src/web/core/src/main/java/org/geoserver/web/data/resource/FeatureResourceConfigurationPanel.java#L132

This in turn might affect all in flight operations attached to the store.

How about we change that a little, so that:

  • The store cleaning, if necessary, is done inside ResourcePool.clear(featureType)
  • The cleaning is done in an intelligent way, and if the store is a ContentDataStore, we only call store.getEntry(name).dispose()?
    (btw, can anyone confirm that is not going to cause issues? any better way to clear the feature type to force the store to re-compute it?)
    And then, have FeatureTypeResource also leverage that code.

In terms of REST API change, I was thinking to mimic the existing reset endpoint, and perform resetting on a feature type if a empty POST is done against the resource, along with a ?reset=true parameter (missing a more significant HTTP method to do so), e.g.:

http://host/geoserver/rest/workspaces/topp/datastores/states_shapefile/featuretypes/states?reset=true

A possible alternative would be to build inside the existing reset path instead, something like a empty POST to:

http://host/geoserver/rest/reset/workspaces/topp/datastores/states_shapefile/featuretypes/states

Opinions?

Cheers

Andrea

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.



Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi


Geoserver-devel mailing list
Geoserver-devel@anonymised.com.366…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Jody Garnett

On Sat, Nov 5, 2016 at 11:58 PM, Ben Caradoc-Davies <ben@anonymised.com>
wrote:

On 04/11/16 23:14, Andrea Aime wrote:

And then, have FeatureTypeResource also leverage that code.
In terms of REST API change, I was thinking to mimic the existing reset
endpoint, and perform resetting on a feature type if a empty POST is done
against the resource, along with a ?reset=true parameter (missing a more
significant HTTP method to do so), e.g.:
http://host/geoserver/rest/workspaces/topp/datastores/states
_shapefile/featuretypes/states?reset=true

This seems the more elegant option to me. Perhaps "?reload=true" as reset
might imply undoing some change? You wrote "reload" in the title and that
seemed clear to me.

Good point.

I wonder if this new functionality has any implications for app-schema
with its cooperating data stores and static data store registry?

Hum... are app-schema delegate stores even exposed in the REST API? I
thought it was managing them internally, without exposing
them to GeoServer?

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

On Mon, Nov 7, 2016 at 7:58 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

I had a JIRA issue on this a while back; when we were looking at how to
let GeoServer know what to do when a new column was added to a postgis
table.
We ended up with a recommendation for something similar to recalculate,
but with different options, including one that would regenerate the cache
of attributes like the GUI (but funding never came through).

- https://osgeo-org.atlassian.net/browse/GEOS-6621
- https://osgeo-org.atlassian.net/browse/GEOS-6636 (sorry the table is
wrecked here)

There is a better way, ContentDataStore has an method to clear is
ContentEntry (which caches the schema at the DataStore level).

Yep, thank you, that indeed looks like a better way.... I am tempted to
implement like this:

store.getEntry(typeName).getState(Transaction.AUTO_COMMIT).flush()

in order to leave be eventual ongoing transactions (otherwise we'd probably
be breaking their accounting/reporting), which are going to die soon
anyways.

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

On 07/11/16 23:54, Andrea Aime wrote:

On Sat, Nov 5, 2016 at 11:58 PM, Ben Caradoc-Davies <ben@anonymised.com>
wrote:

I wonder if this new functionality has any implications for app-schema
with its cooperating data stores and static data store registry?

Hum... are app-schema delegate stores even exposed in the REST API? I
thought it was managing them internally, without exposing
them to GeoServer?

Yes, the delegates are managed internally. I was thinking more of whether there will be any need for an app-schema data store to be reloaded. I cannot think of a use-case. Any mapping changes should be a full REST update and not just a reload.

Kind regards,

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <http://transient.nz/&gt;
New Zealand