[Geoserver-users] Feature: include metadata in shapefile/geopackage WFS output

Hi list, this wraps up some discussion on https://osgeo-org.atlassian.net/browse/GEOS-7972, which was correctly stopped by Andrea, suggestion to open discussion on the mailinglist first.

The issue is that I incidentally get requests to include metadata in shapefiles/geopackages ‘exported’ from a WFS.
That metadata should be extracted from a csw service that is connected to the feature type.
For outputformats GML/json this doesn’t make much sense, but geopackage and shapefile both include a convention to store metadata

Related to metadata on shape files:

  • There seems a convention (coming from esri inc community) to store metadata as {shape}.shp.xml in the zipped archive, but i’m not sure how wide it is supported, and if it may even break shapefile support in other products?

Some challenges here:

  • where to grab the metadata from, on each layer/featuretype a metadata url may be configured, this url would be the most appropriate to grab metadata from, else it could be taken from the embedded geoserver CSW service (using layer metadata). In that case one will have to configure a schema to use.

Jukka suggested a similar feature as mapserver OGR, http://mapserver.org/output/ogr_output.html “wfs_additional_files_in_output”, which may be a valid approach.

Welcoming your thoughts

Regards, Paul.

On 03-02-17 10:24, Paul van Genuchten wrote:

Hi list, this wraps up some discussion

...

Some challenges here:
- where to grab the metadata from, on each layer/featuretype a metadata
url may be configured, this url would be the most appropriate to grab
metadata from, else it could be taken from the embedded geoserver CSW
service (using layer metadata). In that case one will have to configure
a schema to use.

som e other things to think about; what happens when people join featuretypes as part of their request? or change projection or do other things that a (remote) csw is not aware of?...

Hi Paul,
thanks for bringing the discussion in public.
On Fri, Feb 3, 2017 at 10:24 AM, Paul van Genuchten <
paul.vangenuchten@anonymised.com> wrote:

The issue is that I incidentally get requests to include metadata in
shapefiles/geopackages 'exported' from a WFS.
That metadata should be extracted from a csw service that is connected to
the feature type.

For outputformats GML/json this doesn't make much sense, but geopackage and

shapefile both include a convention to store metadata

Related to metadata on shape files:
- There seems a convention (coming from esri inc community) to store
metadata as {shape}.shp.xml in the zipped archive, but i’m not sure how
wide it is supported, and if it may even break shapefile support in other
products?

I'm not familiar with these files. Is the XML file normally containing a
ISO metadata? E.g., something that could literally be downloaded from the
CSW on the fly?
I assume a number of software would just ignore the XML file, but those
that do care, will they get the metadata in a format they expect, or might
they break?

Some challenges here:
- where to grab the metadata from, on each layer/featuretype a metadata
url may be configured, this url would be the most appropriate to grab
metadata from, else it could be taken from the embedded geoserver CSW
service (using layer metadata). In that case one will have to configure a
schema to use.

Agreed. Also, what happens if the CSW cannot be contacted for whatever
reason, or if the call fails? What if the HTTP call hangs up (e.g., due to
a firewall), are there going to be appropriate timeouts?
Thinking out loud, in terms of different approaches, would it make sense to
have some sort of (freemarker?) template that generate the xml file out
only of the internal GeoServer configuration instead?
Would have the advantage of not depending on an external system or optional
GeoServer plugin, although I imagine the resulting description might be a
bit "dry" :-p

Jukka suggested a similar feature as mapserver OGR, http://mapserver.org/
output/ogr_output.html “wfs_additional_files_in_output”, which may be a
valid approach.

Hum... by the looks of the examples these seem like static files instead
right? This could indeed be yet another approach, add a location in the
config to a file
that needs to be included in the response

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

On Fri, Feb 3, 2017 at 3:55 PM, Mark Prins <mc.prins@anonymised.com> wrote:

som e other things to think about; what happens when people join
featuretypes as part of their request? or change projection or do other
things that a (remote) csw is not aware of?...

Joining is probably not a problem right now, only GML and maybe geojson
outputs make sense for it.
The issue is that the join is not returning a flat representation like in
databases, but a set of "tuples"
that contain each N features. Shapefile cannot represent that right now (I
believe it's going to save
the main feature, and the joined ones are going to appear as text derived
from a "toString").

But yes, reprojection might be an issue, one has CSW metadata talking about
the original projection
and the output shapefile can be in another. Same goes for bounds, the
spatial bounds can be reduced
by filtering and the metadata would still talk about the whole dataset
bounds instead.

Unless the "metadata" is generated by a local template that accounts for
the reprojection (and attribute
selection, and bbox filtering, and so on).

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------