[Geoserver-users] Deleting database table through REST API

GeoServer GeoServer User
1

Hi all
When creating a feature type in a database datastore with the REST API, GeoServer will create the corresponding table if it does not exist.
We were wondering if similarly the deletion of a table was also possible when deleting the associated feature type and I found this discussion:
https://sourceforge.net/p/geoserver/mailman/message/36221224/
Does anybody know if a proposal had been submitted? I couldn’t find it on the GitHub wiki but I might have missed it.
Many thanks, Cécile

···

–
camptocamp
Cécile Vuilleumier

Geospatial developer
www.camptocamp.com

2

Hi Cécile,
I don’t see the feature being implemented in the code.

The wiki page you’re looking for is this one:
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-fixes,-improvements-and-new-features-in-GeoServer

Cheers
Andrea

···

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

3

I am open to improvements and addition to the importer API (which offers some control as it is responsible for the automatic creation of tables; it has an opportunity to delete an existing table when replacing contents).

See example https://docs.geoserver.org/latest/en/user/extensions/importer/rest_examples.html#replacing-postgis-table-using-the-contents-of-a-csv-file

···

–
Jody Garnett

4

Hi all,
For the record,I’m utterly against deleting database tables through the geoserver REST API

  • It’s absolutely dangerous
  • The REST API shall with GeoServer “resources”, not infrastructure ones

If I wanted to delete a table through a REST API, I’d better run two queries, one to delete the geoserver FeatureType/Layer,
and another one to delete the database table, for which there are existing solutions, for example PostgREST https://postgrest.org
(not sure if that one lets you drop tables) but you get the idea.

Cheers,
Gabe

···

–
Jody Garnett

Gabriel Roldán

5

Hi,

I was thinking about the same, but if user can do something with the user interface then why not with REST? Both are secured only by the username/password. However, I do not know if it is possible to drop database tables from the UI. I have seen some user requests about cascading deletes so that deleting a layer would also delete the data. I think that is not a good idea.

-Jukka Rahkonen-

···

Lähettäjä: Gabriel Roldan <gabriel.roldan@…84…>
Lähetetty: keskiviikko 14. helmikuuta 2024 17.46
Vastaanottaja: Jody Garnett <jody.garnett@…84…>
Kopio: geoserver-users@lists.sourceforge.net
Aihe: Re: [Geoserver-users] Deleting database table through REST API

Hi all,

For the record,

I’m utterly against deleting database tables through the geoserver REST API

  • It’s absolutely dangerous

  • The REST API shall with GeoServer “resources”, not infrastructure ones

If I wanted to delete a table through a REST API, I’d better run two queries, one to delete the geoserver FeatureType/Layer,

and another one to delete the database table, for which there are existing solutions, for example PostgREST https://postgrest.org

(not sure if that one lets you drop tables) but you get the idea.

Cheers,

Gabe

On Mon, 12 Feb 2024 at 16:25, Jody Garnett <jody.garnett@…84…> wrote:

I am open to improvements and addition to the importer API (which offers some control as it is responsible for the automatic creation of tables; it has an opportunity to delete an existing table when replacing contents).

See example https://docs.geoserver.org/latest/en/user/extensions/importer/rest_examples.html#replacing-postgis-table-using-the-contents-of-a-csv-file

–

Jody Garnett

On Feb 12, 2024 at 1:30:11 AM, Cécile Vuilleumier <cecile.vuilleumier@…1104…> wrote:

Hi all
When creating a feature type in a database datastore with the REST API, GeoServer will create the corresponding table if it does not exist.
We were wondering if similarly the deletion of a table was also possible when deleting the associated feature type and I found this discussion:
https://sourceforge.net/p/geoserver/mailman/message/36221224/
Does anybody know if a proposal had been submitted? I couldn’t find it on the GitHub wiki but I might have missed it.
Many thanks, Cécile

–
camptocamp

Cécile Vuilleumier

Geospatial developer
www.camptocamp.com

Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

–

Gabriel Roldán

6

Hi,

I think it would be helpful to separate the use cases. The original question was about a database table that were created by the REST API when creating a layer. In such a situation it makes sense to delete the table when deleting the layer. Otherwise you could see a scenario were there are a lot of lost tables in the database.

In other situations we have a layer that is wrapped on top of an existing table. In such cases it would be very disruptive to delete the database table when deleting the layer.

I'm in favor of introducing an extra parameter for instance "cascade=true", if the user wants to delete the database table as well. In the last scenario it would also make sense to restrict the database user, that are configured in Geoserver, from making disruptive calls to the database. Like dropping a table.

Regards,
Roar Brænden

14. feb. 2024 kl. 17:42 skrev Rahkonen Jukka <jukka.rahkonen@anonymised.com>:

Hi,

I was thinking about the same, but if user can do something with the user interface then why not with REST? Both are secured only by the username/password. However, I do not know if it is possible to drop database tables from the UI. I have seen some user requests about cascading deletes so that deleting a layer would also delete the data. I think that is not a good idea.

-Jukka Rahkonen-

Lähettäjä: Gabriel Roldan <gabriel.roldan@anonymised.com <mailto:gabriel.roldan@anonymised.com>>
Lähetetty: keskiviikko 14. helmikuuta 2024 17.46
Vastaanottaja: Jody Garnett <jody.garnett@anonymised.com>
Kopio: geoserver-users@lists.sourceforge.net
Aihe: Re: [Geoserver-users] Deleting database table through REST API

Hi all,
For the record,
I'm utterly against deleting database tables through the geoserver REST API
* It's absolutely dangerous
* The REST API shall with GeoServer "resources", not infrastructure ones

If I wanted to delete a table through *a* REST API, I'd better run two queries, one to delete the geoserver FeatureType/Layer,
and another one to delete the database table, for which there are existing solutions, for example PostgREST https://postgrest.org/&gt;
(not sure if that one lets you drop tables) but you get the idea.

Cheers,
Gabe

On Mon, 12 Feb 2024 at 16:25, Jody Garnett <jody.garnett@anonymised.com <mailto:jody.garnett@anonymised.com>> wrote:
I am open to improvements and addition to the importer API (which offers some control as it is responsible for the automatic creation of tables; it has an opportunity to delete an existing table when replacing contents).

See example Importer REST API examples — GeoServer 2.27.x User Manual

--
Jody Garnett

On Feb 12, 2024 at 1:30:11 AM, Cécile Vuilleumier <cecile.vuilleumier@anonymised.com <mailto:cecile.vuilleumier@anonymised.com>> wrote:
Hi all
When creating a feature type in a database datastore with the REST API, GeoServer will create the corresponding table if it does not exist.
We were wondering if similarly the deletion of a table was also possible when deleting the associated feature type and I found this discussion:
Re: [Geoserver-users] How to update/delete postgis tables via REST | GeoServer
Does anybody know if a proposal had been submitted? I couldn't find it on the GitHub wiki but I might have missed it.
Many thanks, Cécile

--
camptocamp
Cécile Vuilleumier
Geospatial developer
www.camptocamp.com <http://www.camptocamp.com/&gt;
_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: Earning Your Support Instead of Buying it
- The GeoServer user list posting guidelines: User group posting guidelines

If you want to request a feature or an improvement, also see this: Successfully requesting and integrating new features and improvements in GeoServer · geoserver/geoserver Wiki · GitHub

Geoserver-users@lists.sourceforge.net <mailto:Geoserver-users@anonymised.comeforge.net>
https://lists.sourceforge.net/lists/listinfo/geoserver-users
_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: Earning Your Support Instead of Buying it
- The GeoServer user list posting guidelines: User group posting guidelines

If you want to request a feature or an improvement, also see this: Successfully requesting and integrating new features and improvements in GeoServer · geoserver/geoserver Wiki · GitHub

Geoserver-users@lists.sourceforge.net <mailto:Geoserver-users@anonymised.comeforge.net>
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Gabriel Roldán
_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: Earning Your Support Instead of Buying it
- The GeoServer user list posting guidelines: User group posting guidelines

If you want to request a feature or an improvement, also see this: Successfully requesting and integrating new features and improvements in GeoServer · geoserver/geoserver Wiki · GitHub

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

7

Hi,

I do not believe that it is generally possible to know if the table was created with REST API or by some other means. It is possible to check the owner of the table and if Geoserver is configured to use a special database account with well thought-out privileges it might be safe to allow to drop tables with REST. However, I believe that in most Geoserver installations the geoserver db user has all too wide privileges, at least if you ask from the DB admins.
If we will have a configuration option “cascade=true” then it should be off by default, and somehow the geoserver admin who turns it on should be made aware of all the consequences.

-Jukka Rahkonen-

···

Lähettäjä: Roar Brænden <roar.brenden.no@…84…>
Lähetetty: keskiviikko 14. helmikuuta 2024 23.29
Vastaanottaja: geoserver-users@lists.sourceforge.net
Kopio: Gabriel Roldan <gabriel.roldan@…84…>; Jody Garnett <jody.garnett@…84…>; Rahkonen Jukka <jukka.rahkonen@…6847…>
Aihe: Re: [Geoserver-users] Deleting database table through REST API

Hi,

I think it would be helpful to separate the use cases. The original question was about a database table that were created by the REST API when creating a layer. In such a situation it makes sense to delete the table when deleting the layer. Otherwise you could see a scenario were there are a lot of lost tables in the database.

In other situations we have a layer that is wrapped on top of an existing table. In such cases it would be very disruptive to delete the database table when deleting the layer.

I’m in favor of introducing an extra parameter for instance “cascade=true”, if the user wants to delete the database table as well. In the last scenario it would also make sense to restrict the database user, that are configured in Geoserver, from making disruptive calls to the database. Like dropping a table.

Regards,

Roar Brænden

  1. feb. 2024 kl. 17:42 skrev Rahkonen Jukka <jukka.rahkonen@…6847…>:

Hi,

I was thinking about the same, but if user can do something with the user interface then why not with REST? Both are secured only by the username/password. However, I do not know if it is possible to drop database tables from the UI. I have seen some user requests about cascading deletes so that deleting a layer would also delete the data. I think that is not a good idea.

-Jukka Rahkonen-

Lähettäjä: Gabriel Roldan <gabriel.roldan@…84…>
Lähetetty: keskiviikko 14. helmikuuta 2024 17.46
Vastaanottaja: Jody Garnett <jody.garnett@…84…>
Kopio: geoserver-users@lists.sourceforge.net
Aihe: Re: [Geoserver-users] Deleting database table through REST API

Hi all,

For the record,

I’m utterly against deleting database tables through the geoserver REST API

  • It’s absolutely dangerous

  • The REST API shall with GeoServer “resources”, not infrastructure ones

If I wanted to delete a table through a REST API, I’d better run two queries, one to delete the geoserver FeatureType/Layer,

and another one to delete the database table, for which there are existing solutions, for example PostgREST https://postgrest.org

(not sure if that one lets you drop tables) but you get the idea.

Cheers,

Gabe

On Mon, 12 Feb 2024 at 16:25, Jody Garnett <jody.garnett@…84…> wrote:

I am open to improvements and addition to the importer API (which offers some control as it is responsible for the automatic creation of tables; it has an opportunity to delete an existing table when replacing contents).

See example https://docs.geoserver.org/latest/en/user/extensions/importer/rest_examples.html#replacing-postgis-table-using-the-contents-of-a-csv-file

–

Jody Garnett

On Feb 12, 2024 at 1:30:11 AM, Cécile Vuilleumier <cecile.vuilleumier@…1104…> wrote:

Hi all
When creating a feature type in a database datastore with the REST API, GeoServer will create the corresponding table if it does not exist.
We were wondering if similarly the deletion of a table was also possible when deleting the associated feature type and I found this discussion:
https://sourceforge.net/p/geoserver/mailman/message/36221224/
Does anybody know if a proposal had been submitted? I couldn’t find it on the GitHub wiki but I might have missed it.
Many thanks, Cécile

–
camptocamp

Cécile Vuilleumier

Geospatial developer
www.camptocamp.com

Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

–

Gabriel Roldán

Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

8

Hi all,
the typical GeoServer approach is not to be opinionated about things.
“Proudly letting users free since 2002” could be our slogan… want to be strictly OGC compliant? We have a checkbox for you.
Want to do things in a more convenient way and the hell with standard strictness? We also support that.

Table creation is already allowed both by the importer (which also imports the data), and by the GUI/REST API for table creation
(WFS-T should then be used for data import):

image.png

image.png

Having a delete would make things symmetric, and I agree that cascade should be explicitly added for that
to happen.

That said, I also hear the voice of those claiming “geoserver should not be used to manage data”.
For that, we should have a simple warning in the production section: “GeoServer offers ability to create
and drop tables through UI and REST API, if this is not your intent, please lock down the database
grants for the account used by GeoServer”… which is generally speaking also good IT hygiene in general.
(don’t give accounts more rights than necessary).

Cheers
Andrea

···

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

9

I like Andrea’s take but Gabriel made a very good point.
I agree with Gabriel that certain tasks should stay outside GeoServer but reality is that we have a number of users which have very small infrastructures and they cannot afford having pipelines to perform some tasks outside of it and they simply expect GeoServer to do such tasks for them.

I’d say that whenever possible and not difficult to maintain we should try to give them tools to ease their work (and simplify GeoServer adoption).

image.png

image.png

···

Regards,

Simone Giannecchini

Online training classes for GeoNode, GeoServer and MapStore from the experts!
Visit https://www.geosolutionsgroup.com/professional-training/ for more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director GeoSolutions Italy
President GeoSolutions USA

phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928
US: +1 (845) 547-7905

http://www.geosolutionsgroup.com
http://twitter.com/geosolutions_it

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

10

Indeed good points about ease of use.
Still, I’d add to the danger list:

  • you can’t guarantee the atomicity of the operation

  • in general terms it is not the exact reverse of the create feature type functionality. A single db table may be used by several feature types across different workspaces even

  • Flag parameters are bad design, increases complexity and understanding, breaks a couple principles. Worse case scenario it can’t be called cascade=true, the workspace delete endpoint already uses it with different semantics, unless you want to delete all tables which would be catastrophic IMHO

My compromise solution would then be to have a separate operation to delete the table:

  • keep delete feature type atomic and single purpose.
  • the delete table endpoint would be atomic and fail if the table is used by any feature type. simple.

question arises of how to determine which table to delete once you deleted the FeatureType. I guess it should be an operation of the DataStore and not of the FeatureType, and use the FeatureType’s nativeName to distinguish?

2c.
Gabe

image.png

image.png

···

Regards,

Simone Giannecchini

Online training classes for GeoNode, GeoServer and MapStore from the experts!
Visit https://www.geosolutionsgroup.com/professional-training/ for more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director GeoSolutions Italy
President GeoSolutions USA

phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928
US: +1 (845) 547-7905

http://www.geosolutionsgroup.com
http://twitter.com/geosolutions_it

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

Gabriel Roldán

11
  • Flag parameters are bad design, increases complexity and understanding, breaks a couple principles.

I mean “increases complexity and difficults understanding”, of course.

question arises of how to determine which table to delete once you deleted the FeatureType. I guess it should be an operation of the DataStore and not of the FeatureType, and use the FeatureType’s nativeName to distinguish?

Which makes me thing, if I’m not mistaken, there’s no REST endpoint to gather the list of a DataStore’s native feature types. Like if you wanted to create a rich client app using strictly the REST API, how would you list the store’s available feature types?

with that in mind:
GET …//stores//nativeFeatureTypes would give you a list of raw type in the store
GET …//stores//nativeFeatureTypes/ its raw FeatureType

DELETE …//stores//nativeFeatureTypes/ would drop the native FT. Possibly failing if it’s in use, or cascade deleting published feature types for that native type (table)

opinions?

12

Hi,

Good points, Gabriel.
I support having a little more explicit end-points for the different operations.

Regards,
Roar Brænden

  1. feb. 2024 kl. 15:00 skrev Gabriel Roldan <gabriel.roldan@anonymised.com>:
  • Flag parameters are bad design, increases complexity and understanding, breaks a couple principles.

I mean “increases complexity and difficults understanding”, of course.

question arises of how to determine which table to delete once you deleted the FeatureType. I guess it should be an operation of the DataStore and not of the FeatureType, and use the FeatureType’s nativeName to distinguish?

Which makes me thing, if I’m not mistaken, there’s no REST endpoint to gather the list of a DataStore’s native feature types. Like if you wanted to create a rich client app using strictly the REST API, how would you list the store’s available feature types?

with that in mind:
GET …//stores//nativeFeatureTypes would give you a list of raw type in the store
GET …//stores//nativeFeatureTypes/ its raw FeatureType

DELETE …//stores//nativeFeatureTypes/ would drop the native FT. Possibly failing if it’s in use, or cascade deleting published feature types for that native type (table)

opinions?

Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

13

I mean “increases complexity and difficults understanding”, of course.

But parameters are already widely used in the GeoServer REST API, and “cascade” is used in other places as well.
This would be breaking convention, making the API harder to use for those that are already used to it.

Don’t get me wrong, I know the GeoServer REST API is old and would need a rewrite, but that’s the key point, a rewrite would be the time to break
compatibility and adopt new ways of doing things.

question arises of how to determine which table to delete once you deleted the FeatureType. I guess it should be an operation of the DataStore and not of the FeatureType, and use the FeatureType’s nativeName to distinguish?

The REST API only returns configured feature types by default. There is (guess what?) a parameter in the “featuretypes” resource, called “list”, that can take
3 different values:

  • “configured” (default if not specified): only lists the configured feature types (links to the feature type info resource)

  • “available”: returns the native feature types not yet configured (mind, only their names)

  • “available_with_geom”: same as above, bon only spatial ones

  • “all”: returns all of them, configured or available (again, just names)

The FeatureTypeController delete mapping already has a “recurse” flag to delete layers while the feature type is removed.
Now here there is a risk of confusion between “recurse” and “cascade”, a “removeData” flag would probably avoid confusion.

Cheers
Andrea

···

GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

14

Two wrongs don’t make a right, IMHO.
I’d rather break convention than introduce such a dangerous parameter to an existing API endpoint and change its semantics

I’m not talking about a rewrite of the REST API, but a new verb to the existing API. Don’t see how that’d make it harder for the people that’s used to it, when it doesn’t even exist. Used the the rest api having all sorts of parameters I take it. So how much harder is it when I ask “how do I delete a table in my database?”- oh, just need to add this param to DELETE …/stores/mystore/featuretypes/xxx

  • or, oh, just need to call DELETE …/stores/mystore/drop/xxx

For the former, you need to update the documentation saying if you pass this, then it does this, if you pass this other parameter then it does this other thing. Beware to check all your scripts, because if you accidentally leave this parameter on, your company database will be destroyed.
As opposed to not changing the current api and adding an endpoint that says:
This will delete the database table. Fails in case there’s an existing FeatureType using it.

My point is this deserves its own endpoint. Way more explicit and less error prone than a new param to the current delete featuretype operation, which would change its semantics so drastically.

In any case, that’s just my opinion.

Cheers,
Gabe

···

camptocamp
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

Gabriel Roldán
Geospatial Developer

GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

15

I’m in disagreement with both the direction taken and the attitude shown, best to put an end to it and make it go through a formal GSIP instead (when someone has the resources to spend on both the discussion and implementation of it)

Regards
Andrea

···

camptocamp
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

Gabriel Roldán
Geospatial Developer

GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

16

I sorry you feel that way Andrea.
I’ve a strong opinion on the matter, yes. How relevant is it? as much as anyone else’s.
Peace.
Gabe

···

camptocamp
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

Gabriel Roldán
Geospatial Developer

camptocamp
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

Gabriel Roldán
Geospatial Developer

GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

17

Cécile,

My customers would find this concept useful, and I would support you in making a proposal.

I like the idea of making a new REST API endpoint to avoid overlap and accidental use. Also gives an opportunity for and admin setting to disable for those inclined.

···

Jody Garnett

18

Ok,
let’s start over with more context. In particular, let’s start with the existing API and how it works in terms of “native” resources, and what “dangerous” functionality is already there.

List of native resources
How does one get a list of resources (feature types, coverages, wms and wmts layers), all GET requests here:

  • /workspaces/{workspaceName}/datastores/{storeName}/featuretypes?list=all/available (either all of them, or the ones non configured)

  • /workspaces/{workspace}/coveragestores/{store}/coverages?list=all (returns both configured and non configured, no option implies only configured)

  • /workspaces/{workspace}/wmsstores/{wmsstore}/wmslayers/{wmslayer} → no ability to list available layers here!

  • /workspaces/{workspace}/wmtsstores/{wmtsstore}/layers?list=available/configured

When the parameters have the same name, they do the same thing, but the availability of them is spotty. The only type of native resource that cannot be enumerated in the WMS layer one.

Exploring a non configured native resource

This does not seem to be available for any type of native resource. This would be useful, although not part of Cecile’s original request.

Creation of new native resources
How does one create a new native resource that was not there:

  • POST /workspaces/{workspaceName}/datastores/{storeName}/{method}.{format}?target=<targetSTore). This uploads a file (e.g. shapefile) and dumps it on the server. If the targetStore is set, a createSchema will be run and the data dumped in the target store. If the target schema is already present and “overwrite=true” is part of the request, all existing data is deleted and replaced with the new data.

  • POST /workspaces/{workspaceName}/datastores/{storeName}/featuretypes, posting a definition that is not yet there. This creates an empty table/file.

  • POST /workspaces/{workspaceName}/coveragestores/{storeName}/{method}.{format} will upload some raster data or mosaic definition. If the mosaic is configured to work with database, it will automatically create the necessary index tables.

  • POST /workspaces/{workspaceName}/coveragestores/{storeName}/{method}.{format}. This allows one to configure a new entry in a mosaic, potentially with data upload. While it does not create a new layer per se, it makes new data available and can write data on the server side.

For WMS/WMTS layers there is no way to create a layer in the remote server. Interestingly enough, the PostGISDataStoreFactory has a CREATE_DB_IF_MISSING parameter that will run a “CREATE DATABASE” if the target database is not there. Database creation has been available since 2014.

Removal of native resources
How does one clean a native resource that’s no longer needed:

  • For feature types, we already know this is not available. The same goes for vector stores. The story here is asymmetric, one can create via FeatureType/DataStore controllers, but not remove.
  • For WMS/WMTS layers, we can’t control the remote server.
  • DELETE /workspaces/{workspaceName}/coveragestores/{storeName}?deleteType=none/all/metadata. With “metadata” the configuration files are all removed (e.g., the mosaic configuration files), with “all” the actual raster data is removed as well
  • DELETE is also available at the single mosaic file entry as well, same approach, one can either the registration in the mosaic, or both the registration and the actual data.
    Summary

The existing REST API abilities are scattered, but generally allows creation of both database, new tables, new data.
Removal is fully developed only on raster data (typical use case of having to maintain a time window of data online, in environments where standing up a separate process to perform data management is seen as overhead).
I hope this clarifies why I see it as more natural to make the API symmetric in the feature type case, rather than creating a new variant that is for feature types only.
Or do you want to make it consistent and adopt the same for coverages as well? What about the existing functionality, should it be dropped? We don’t have a history of deprecating REST APIs but I’m open to discussion.

Dangers of using the wrong API
Here are some considerations on “having a new API is less dangerous than a parameter”.
I’ìm trying to put myself in the place of someone writing a REST script. If one needs to just remove feature types, they will use:

DELETE /workspaces/{workspaceName}/datastores/{storeName}/featuretypes/{typeName}

If they never need to drop tables, there is no chance that they’ll add a “purgeData=true” by mistake (pick whatever parameter name you want, as long as it’s clearly separate from “cascade” and clearly indicates data destruction).
If they instead are playing with the idea, and are still undecided on what to do, they will likely have something like this, with the parmeter commented out but ready to be put back in.
I’m simulating with color a bit of syntax highlighting, and keeping cascade=true in all calls because we most likely want to remove the associated LayerInfo as well.

DELETE /workspaces/{workspaceName}/datastores/{storeName}/featuretypes/{typeName}?cascade=true #?dropData=true

If instead we have a separate API, they will likely have this:

DELETE /workspaces/{workspaceName}/datastores/{storeName}/featuretypes/{typeName}?cascade=true

DELETE /workspaces/{workspaceName}/datastores/{storeName}/nativeFeaturetypes/{typeName}?cascade=true

I have a hard time figuring out how one is more dangerous than the other.
Going by personal experience, I would be a bit more prone to make an error with the second case, because the difference between one and the other lies in the middle of the path rather that being separate at the end… but it’s probably just me, I tend to focus and structure and glance over the text, rather than actually reading it word by word.

Small data management side note
Dealing with data import/management/removal is inherently dangerous no matter what tool is used.
The documentation for the tools allowing it should be clear, and indications on proper security setup is also beneficial.

Conclusion
Please, let’s try to go towards a direction of more API consistency, rather than less of it.
So I’m ok with the a “native” API, but at least let’s have it consistently
available across the board, rather than having a “feature type exception”.
Let’s also have a plan on how to handle the inevitable functional redundancy (deprecation? removal?)

Andrea

···

camptocamp
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

Gabriel Roldán
Geospatial Developer

GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

19

Hi Andrea,

Thanks for this valuable break down.

Everything you mention makes sense, and I think we’re getting to the root of our disagreement.
Believe me I absolutely understand the convenience of just adding a parameter to the existing delete featuretype api. Heck, we would have saved all the time invested in this discussion an my customers would get what they want, very low hanging fruit.

Yet I’m getting involved, just like you, thinking of the bigger picture and the general good for geoserver.

So let me try to distill my reasoning about this.

The create feature type and for instance, all (most?) of the cataloginfo related endpoints in the core api, are RESTful-ish, in the sense they deal with REpresentational State Transfer of Catalog resources. As such, they are intend to be atomic and single purpose.

For the specific case of the create featuretype api, creating the schema on the target store is a desired byproduct, as it’s currently designed, and there’s no argument against its convenience.

One can say the operation’s single purpose is to create a FeatureTypeInfo, and as a convenience it’ll create a LayerInfo and the underlying native schema.
The main point being it’s atomic, when it returns, the featuretype is either created or it isn’t. Worse case scenario, something failed and the system left a dangling schema in the backing store, but no harm’s done to the system.

Once create featuretype succeeded, if eventually it created a native store featuretype, now that’s a separate entity on its own right, on a different domain than the Catalog objects. So much that it can be used by several other published featuretypes. Even in SQL-view published types.

Now, when it comes to purging the (or a, for instance) native featuretype, we’re no longer talking about the core api’s delete featuretype semantics. Those deal with removing a FeatureTypeInfo from the Catalog. As a by-product it’d delete the associated LayerInfo. Again, it should be atomic.

What we’re talking about now, instead, is a use case specific API. Why:

  • because it has to deal with (destructive) operations at more than one domain level (the catalog object and the native type)
  • because it has to account for other published FeatureTypes that might be depending on the same native FeatureType. In this case a decision has to be made on whether also delete those catalog objects or fail the operation.
  • because it can’t be atomic. At the end of the operation, both the published and native FTs shall be removed, or none. The Catalog doesn’t handle transactions, and also not all DataStores do. Much less distributed ones. Hence, if an error occurs at any stage, you end up with an inconsistent state: the database table was removed but you couldn’t remove the pusblished type, or the other way around. In the former case, you can’t recover the data. In the later, the state is intermediate, and the operation didn’t achieve its goal.

So that’s IMO the distinction. Not about parameters or path variables, which are essentially the same thing technically speaking.
I’ve nothing against operation parameters as long as they’re input values to the operation. What I’m against is flags that change its semantics.

It is so much very true that as it stands today, the api parameters allow to support a wide range of use cases. But trying to be so flexible that a single endpoint supports all possible use cases, to me, is a mistake.

From the above reasoning my conclusion is the ability to delete a published feature type, alongside the underlying data and schema, and possibly any other published feature type depending on the native schema, is a use case and not a REST operation, and hence deserves its own use-case specific api endpoint, with its own request and response bodies. That or, you oughta deal with the use case client-side, and call separate endpoints for the different domains.

Finally, in your current-api-breakdown reply, you mention the only existing related endpoint is to delete also the data for a coverage:

DELETE /workspaces/{workspaceName}/coveragestores/{storeName}?deleteType=none/all/metadata. With “metadata” the configuration files are all removed (e.g., the mosaic configuration files), with “all” the actual raster data is removed as well

Well, this is the perfect example of what I’m proposing. Note how it’s an operation on the CoverageStore and not on the Coverage.
I concede that’s probably related to the fact that CoverageStores support a single Coverage?

Point is, this is definitely a use case driven api end point, not the REST endpoint for CoverageInfo. And what I’m proposing is just like that.

I think that exposes my point of view in a more civil way, and yes, we’ll probably have to go through the GSIP process.

Best regards,

···

camptocamp
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

Gabriel Roldán
Geospatial Developer

camptocamp
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

Gabriel Roldán
Geospatial Developer

GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail

20

Believe me I absolutely understand the convenience of just adding a parameter to the existing delete featuretype api. Heck, we would have saved all the time invested in this discussion an my customers would get what they want, very low hanging fruit.

Sorry Gabriel, I don’t think you understood then… It’s not at all about convenience, but about continuity and symmetry.
The existing API has been around for a while, has been used, it has a counterpart in coverages, and it’s just lacking the
method to perform the delete along the same lines.

I see where you’re coming from, but to me, that’s a new REST API (v2). If there is a separate API to handle native types,
so be it, but then the old API should be deprecated, and symmetry restored, that is, the coverages world should also have one.

The create feature type and for instance, all (most?) of the cataloginfo related endpoints in the core api, are RESTful-ish, in the sense they deal with REpresentational State Transfer of Catalog resources. As such, they are intend to be atomic and single purpose.

Sorry, we are again not on the same page. Atomicity is a very useful property, but one that the existing REST API does not
offer in general, and when it does, it is by accident. The reason is simple: the service and catalog models lack the concept of
a transaction that can be started at the beginning of a request, and committed atomically at the end, or reverted fully if anything wrong happens.
The current REST API has no such promise, because we lack the underlying machinery to satisfy the promise.

Now, I guess that Cloud GeoServer with the PostgreSQL configuration backend might wrap operations in a transaction, which is fine,
but not something we can claim for the API in general (not against the random GeoServer).
Generally speaking, I care about what GeoServer can do, if we can help downstreams projects it’s even better, but not at the
expense of having an inconsistent situation when one can “enter the house through the door, but be forced to leave through a window”
(load data through the existing feature resources, but remove it through the native type resource).

I’d be more inclined to have delete as a parameter, and then in addition, a native resource API as well (which has the merit
of describing types not currently published). A user can then decide which path they prefer: multiple alternatives are
nothing new, for large data uploads using GeoServer only, the importer API is the preferred alternative anyways, due to its async operation support.
The concept of “native resource” would likely have to be explained in detail, we already have a layer and feature type, the native bit would introduce
a third level (which is natural to those familiar with the internals, but not so sure about random Joe).

For the specific case of the create featuretype api, creating the schema on the target store is a desired byproduct, as it’s currently designed, and there’s no argument against its convenience.

That’s your interpretation of it, the API evolved out of practical considerations, not sure how much up front design was put into it.

One can say the operation’s single purpose is to create a FeatureTypeInfo, and as a convenience it’ll create a LayerInfo and the underlying native schema.
The main point being it’s atomic, when it returns, the featuretype is either created or it isn’t. Worse case scenario, something failed and the system left a dangling schema in the backing store, but no harm’s done to the system.

If one uploads in replace mode, a lot of damage can be done… the old data is removed before adding the new one.
All one needs is to upload a file that can be self described properly, but cannot be properly read (corrupted shapefile for example).

  • because it can’t be atomic. At the end of the operation, both the published and native FTs shall be removed, or none. The Catalog doesn’t handle transactions, and also not all DataStores do. Much less distributed ones. Hence, if an error occurs at any stage, you end up with an inconsistent state: the database table was removed but you couldn’t remove the pusblished type, or the other way around. In the former case, you can’t recover the data. In the later, the state is intermediate, and the operation didn’t achieve its goal.

Skipped over and came back here. All reasoning based on atomicity guarantees is faulty, it cannot be guaranteed in general unless we change the Catalog and Geoserver interfaces.

I’ve nothing against operation parameters as long as they’re input values to the operation. What I’m against is flags that change its semantics.

POST creates (configuration and data) and DELETE destroys (configuration and data). It’s symmetric.
If you want to remove the ability to delete data, you need to get rid of the ability to create it as well.

Finally, in your current-api-breakdown reply, you mention the only existing related endpoint is to delete also the data for a coverage:

DELETE /workspaces/{workspaceName}/coveragestores/{storeName}?deleteType=none/all/metadata. With “metadata” the configuration files are all removed (e.g., the mosaic configuration files), with “all” the actual raster data is removed as well

Well, this is the perfect example of what I’m proposing. Note how it’s an operation on the CoverageStore and not on the Coverage.
I concede that’s probably related to the fact that CoverageStores support a single Coverage?

No, it’s meant to go and destroy all coverages and associated data contained in the mosaic, if called that way.
If the same is called on the single coverage, it will delete only the data files associated to that coverage, and that coverage alone.
E.g., if a NetCDF file happens to offer content for multiple coverages, it will be removed only when the last coverage referencing it
it is removed as well.

These APIs are meant to be used by data managers that pay attention to what they do (“admin” in GeoServer is tha same
as “root” on Linux, all powerful).

A separate resource makes a difference only in your mind, in my experience I’m just as easily going to make a mistake in the path
or in the query string (see example in my previous mail).

I think that exposes my point of view in a more civil way, and yes, we’ll probably have to go through the GSIP process.

Thank you for the explanation, unfortunately it did not change my mind.
Further discussion is pointless, let’s get the PSC members to vote, and take responsibility for the result, whatever it is.
I’m happy to let the proposal pass anyways, if the rest of the PSC is happy with the direction you’re suggesting, but I want my disagreement to be on record.

Cheers
Andrea

···

GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions Group
phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

https://www.geosolutionsgroup.com/

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail