Jody,
Even though I am not entitled to vote, I’d like to point out that experiences from Testbed 17 clearly show that it is not wise to use GPKG as output format to synchronous WMS and WFS requests issued by users.
Each request binds resources until the GeoPackage is complete. Cancellation is not possible. If the GeoPKG production takes too long, the user may “click” again and again and again and thereby cause binding of resources for completing the same request that perhaps is still in progress (but defunct). The fact that the client is no longer able to consume the response (either connection timeout or user cancelled the request) is found out only after the GeoPackage is fully produced and to be send to the client (IO Exception as socket is closed). Until then, all necessary resources of the thread (CPU, Memory, diskspace) are bound to produce a GeoPackage for nothing…
In addition we found the GeoPackage axis order issue (as you point out in #3) and reported that as bug: https://osgeo-org.atlassian.net/browse/GEOT-7011
Best
Andreas
From: Jody Garnett jody.garnett@anonymised.com
Date: Wednesday, 5. January 2022 at 03:16
To: Simone Giannecchini simone.giannecchini@anonymised.com
Cc: Geoserver Devel geoserver-devel@lists.sourceforge.net
Subject: Re: [Geoserver-devel] GSIP-206 Promote GeoPackage to extension
Simone, this is the definition of slow moving and not urgent. I first asked if this was a good idea around 2 years ago as a customer is interested. Procurement moves slowly, funding has now come through to do this activity. The customer has been using geopackage WFS output for at least three years. Indeed this funding is as a result of a review of the community modules the customer is using in production, and asking if they would be interested in helping it be cleaned up and made into an official extension.
While I do not have a sizable number of clients, once this functionality has been cleaned up I will be adding it to the geocat enterprise products for all our customers (presently it is an option by request).
Let’s review the checklist:
1. The module has at least a “handful” of users.
I have three users of this functionality, only one I would consider as using it in production.
2. The module has a designated and active maintainer
I offered to join Andrea on this (on the assumption the functionality can be cleaned up and be included in geocat enterprise products).
3. The module is considered “stable” by the majority of the PSC
I checked in with this on november/decemeber expecting wfs and wps to be stable and wms output format to be removed. Andrea asked me to revise the proposal to include gs-wms.
So far my personal testing has been mixed, EPSG:4326 output is coming out in Y/X order which does not match up with the specification:
The axis order in WKB stored in a GeoPackage follows the de facto standard for axis order in WKB and is therefore always (x,y{,z}{,m}) where x is easting or longitude, y is northing or latitude, z is optional elevation, and m is optional measure. This ordering explicitly overrides the axis order as specified in the SRS metadata, applying Case 4 from OGC 08-038r7, Revision to Axis Order Policy and Recommendations[K11]. This was done to maintain consistency with previous implementations of WKB that predated the OGC policy.
The above indicates additional QA is needed. You can also see the proposal where I noted frustration with a couple design decisions.
4. The module maintains 40% test coverage
The coverage looks good, but I have not measured it.
5. The module has no IP violations.
So far everything seems to be original work, with links to OGC where appropriate. If something comes up during the activity I will let you know.
6. The module has a page in the user manual
Not directly useful as the documentation will be distributed across several pages:
I also note geosolutions has training materials https://docs.geoserver.geo-solutions.it/edu/en/wps/geopackage_output.html
7. The maintainer has signed the GeoServer Contributor Agreement
OSGeo has a signed CLA from both myself and GeoCat BV.
–
Jody Garnett
On Tue, 4 Jan 2022 at 14:23, Simone Giannecchini <simone.giannecchini@anonymised.com> wrote:
Good Morning Jody,
I am not too inclined on having the gpkg output jump from community to core for WMS and WFS.
The process we have in place is there to exactly prevent something like this from happening because “someone needs it urgently”.
I mean, have you been using them in production enough to be confident with them? Do you already have a sizable number of clients using the extensions so we can trust them? I guess not given what you said above…
For the moment my vote is a -1 on this, but I am happy to hear your thoughts on my points above.
Regards,
Simone Giannecchini
GeoServer Professional Services from the experts!
Visit http://bit.ly/gs-services for more information.
Ing. Simone Giannecchini
@simogeo
Founder/Director GeoSolutions Italy
President GeoSolutions USA
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928
http://www.geosolutionsgroup.com
http://twitter.com/geosolutions_it
This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
On Fri, Dec 31, 2021 at 1:50 AM Jody Garnett <jody.garnett@anonymised.com> wrote:
Proposal is renamed.
With respect to gs-wps module I would like to see the matching gt-wps unsupported module which forms its foundation cleaned up (finally). Something we can discuss in the new year.
–
Jody Garnett
On Thu, 30 Dec 2021 at 07:40, Jody Garnett <jody.garnett@anonymised.com> wrote:
Thanks Andrea, I will rename the proposal.
I have capacity to support the wps module on this one (as indeed a customer is funding this activity).
Jody
On Thu, Dec 30, 2021 at 5:11 AM Andrea Aime <andrea.aime@anonymised.com> wrote:
Hi Jody,
checking the proposal, I believe the title is misleading, it seems to suggest a classic module move from community to extension, as is… which does not match the actual proposal.
The actual proposal is:
· Fold the WFS output format gs-wfs, the WMS output format in gs-wms, hence, move these two bits in core
· Fold the WPS process in gs-wps-core
About the move to cover of the WMS/WFS output formats, from a technical point of view I’m not concerned:
· The WFS output format will eventually break for large outputs due to HTTP time outs (needs to be written on disk first, streamed back once complete), but the same already happens for zipped shapefiles, so nothing really new
· The WMS output format now honors the rendering time outs, so it should be fine
I should however point out that the core of GeoServer is “maintained by the PSC” so the rest of the PSC should be comfortable having that code in core, and realize the associated obligation.
About the move of the WPS process to wps-core, I’m also personally not deeply concerned, if the documentation
is very clear about the experimental extensions baked into the process (so doc updates are needed).
I’m however noting the code moving also moves its maintainership from “nobody” to me (the WPS module maintainer), which I’m not too fond of [1].
Since you’re making the proposal, I’d like you to step up as co-maintainer of the code you’re trying to push up. For the core bits, as a PSC member, you’re taking that
responsibility automatically anyways. Please step up to co-maintain the processes once moved in gs-wps-core.
Cheers
Andrea
[1] Due to both project and business obligations I’m responsible for way too many modules already,
something which is too big of a onus on my shoulders, and a big project liability in case I get sick
or decide to leave. It’s something we’ll have to address, possibly sooner rather than later.
On Thu, Dec 30, 2021 at 5:28 AM Jody Garnett <jody.garnett@anonymised.com> wrote:
Please have a look at
https://github.com/geoserver/geoserver/wiki/GSIP-206 which outlines moving different sections of the geopackage community module into the appropriate core module: gs-wfs, gs-wms, and gs-wps.
The proposal is solid, please make note of the backwards compatibility section which proposes calling out wps geopackage supports for not yet finalized extensions. While there is nothing wrong with having geoserver specific extensions they should be documented as such (and marked as in progress if they are chasing a moving target). If folks feel strongly about documentation not being enough warning I can look at leaving these geopackage extensions as an optional install.
Jody
–
–
Jody Garnett
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
–
Regards,
Andrea Aime
==
GeoServer Professional Services from the experts!
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions Group
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928
https://www.geosolutionsgroup.com/
http://twitter.com/geosolutions_it
Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail
–
–
Jody Garnett
Geoserver-devel mailing list
Geoserver-devel@anonymised.comforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
_______________________________________________ Geoserver-devel mailing list Geoserver-devel@anonymised.comourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
).