[Geoserver-users] GML QName issues with PostGIS feature store

Hi,

You wrote We use UUIDs to identify our datasets

Does it mean that you have also typenames like 402b28fe-af35-463f-a78f-47f121cea236 in your service? If you have then they are invalid as well because names of the feature types are QName. One old discussion about the topic is here http://osgeo-org.1560.x6.nabble.com/Re-Geoserver-devel-wms-getCapabilities-special-characters-td5160099.html.

Renaming the PostGIS columns is pretty easy to do with SQL View layer as Andrea told, or by creating the view into the database and publishing the view with Geoserver.

Special issue with Geoserver is that layers are usually automatically published as WMS and WFS or WCS with the same layer/featuretype/coverage name and while WMS and WCS 1.0.0 accepts anything as layer name WFS and WCS 2.0 are strict.

-Jukka Rahkonen-

···

Matthias Müller wrote:

Hi Andrea,

thanks for the quick reply.

Just for the record: Correct behaviour towards the client also matters in option (2). Possibilities for handling wrong GML are:

2a) return an HTTP error (fail early)
2b) hide invalid layers in Capabilities
2c) disable GML output on a particular WFS endpoint if one of the layers is invalid (contradicts WFS spec!)
2d) disable WFS endpoint until data becomes valid

All of these are far from ideal, so option (2) might be the least favorable solution.

Kind regards,
Matthias

Am 15.06.2016 um 11:14 schrieb Andrea Aime:

Hi Matthias,

I believe this is intended behavior, it’s not up to the data store code to police attribute names

and ids. E.g., if you were just using this layer for WMS, there would not be any problem,

serving also WFS is your choice, many do, but not everybody.

We have been over this a few times without any good consensus (and more importantly,

without any funding to take action on it), various positions are:

  • Do not allow publishing if the data is not compatible with all services

  • Warn the admin if a layer is not compatible with some service restrictions

  • Perform on the fly mappings to avoid this issue, with different mappings per service

The quickest solution for you is probably to setup a sql view that amends the names and

ids so that they are compliant. This works as long as you don’t have to edit via WFS-T of course.

Cheers

Andrea

On Wed, Jun 15, 2016 at 10:58 AM, Matthias Müller <matthias_mueller@…4476…> wrote:

Hello everyone,

We are running a Geoserver instance with Postgres/PostGIS backend. Everything
seems to work fine so far, except for WFS with GML.

We have two specific issues here:

  1. The auto-generated feature IDs (see [1]) do not comply with the QName
    convention. We use UUIDs to identify our datasets and Geoserver seems to uses
    this UUID directly to generate Feature IDs. Since QNames must not start with a
    number, we get invalid GML for roughly 10 out of 16 datasets. Most
    Validators/Clients we have tested fail to parse this GML (QGIS among them).

  2. Geoserver uses Postgres column names as feature attribute names. Since
    Postgres column names are not restricted to QNames (e.g. “12345”, “km/s” are
    valid column names), we sometimes run into similar issues here.

Is this intended behavior, i.e. is the dataset provider responsible for
ensuring QName compliant attribute names, or is it a bug?

Kind regards,
Matthias

p.s.: Geoserver version is 2.8.3

[1]:

gml:featureMember
<ckandev:402b28fe-af35-463f-a78f-47f121cea236
fid=“402b28fe-af35-463f-a78f-47f121cea236.1”>
ckandev:X51.003874</ckandev:X>
ckandev:Y13.726043</ckandev:Y>
ckandev:Timestamp2012-07-20T:10:00GMT+1</ckandev:Timestamp>
ckandev:PlaceBannewitz</ckandev:Place>
ckandev:Zn_1000_-_400___micro_g_per_g_127.3936706</ckandev:Zn_1000_-400___micro_g_per_g>
ckandev:Zn_400_-_100___micro_g_per_g_418.9342066</ckandev:Zn_400_-100___micro_g_per_g>
ckandev:Zn_100_-_63___micro_g_per_g_743.5718278</ckandev:Zn_100_-63___micro_g_per_g>
ckandev:Zn_63_-_0.45___micro_g_per_g_528.2146161</ckandev:Zn_63_-0.45___micro_g_per_g>
ckandev:Zn_SUMM___micro_g_per_g_377.8245751</ckandev:Zn_SUMM___micro_g_per_g_>
ckandev:Cu_1000_-_400___micro_g_per_g_48.0749849</ckandev:Cu_1000_-400___micro_g_per_g>
ckandev:Cu_400_-_100___micro_g_per_g_112.1762429</ckandev:Cu_400_-100___micro_g_per_g>
ckandev:Cu_100_-_63___micro_g_per_g_164.8965903</ckandev:Cu_100_-63___micro_g_per_g>
ckandev:Cu_63_-_0.45___micro_g_per_g_151.0506145</ckandev:Cu_63_-0.45___micro_g_per_g>
ckandev:Cu_SUMM___micro_g_per_g_105.0860617</ckandev:Cu_SUMM___micro_g_per_g_>
ckandev:Cd_1000_-_400___micro_g_per_g_0.056641806</ckandev:Cd_1000_-400___micro_g_per_g>
ckandev:Cd_400_-_100___micro_g_per_g_0.166999107</ckandev:Cd_400_-100___micro_g_per_g>
ckandev:Cd_100_-_63___micro_g_per_g_0.264253773</ckandev:Cd_100_-63___micro_g_per_g>
ckandev:Cd_63_-_0.45___micro_g_per_g_0.278181578</ckandev:Cd_63_-0.45___micro_g_per_g>
ckandev:Cd_SUMM___micro_g_per_g_0.1663264</ckandev:Cd_SUMM___micro_g_per_g_>
ckandev:Shape
<gml:Point srsName=“http://www.opengis.net/gml/srs/epsg.xml#4326”>
<gml:coordinates xmlns:gml=“http://www.opengis.net/gml
decimal=“.” cs=“,” ts=" ">13.726043,51.003874</gml:coordinates>
</gml:Point>
</ckandev:Shape>
</ckandev:402b28fe-af35-463f-a78f-47f121cea236>
</gml:featureMember>


What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity planning
reports. http://pubads.g.doubleclick.net/gampad/clk?id=1444514421&iu=/41014381


Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

==

GeoServer Professional Services from the experts! Visit

http://goo.gl/it488V for more information.

==

Ing. Andrea Aime

@geowolf

Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)

phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


Hi Jukka,

UUIDs should be compliant with the general feature model for both feature identifiers and feature attributes. They just invalid with certain encodings such as GML (and WFS, which depends on GML encoding). With software, that supports multiple encodings, this raises questions regarding expected preconditions and/or conventions for on-the-fly mappings.

Best,

Matthias

From: Rahkonen Jukka (MML) [mailto:jukka.rahkonen@anonymised.com]
Sent: Wednesday, June 15, 2016 2:42 PM
To: Matthias Müller; GeoServer Mailing List List
Subject: Re: [Geoserver-users] GML QName issues with PostGIS feature store

Hi,

You wrote We use UUIDs to identify our datasets

Does it mean that you have also typenames like 402b28fe-af35-463f-a78f-47f121cea236 in your service? If you have then they are invalid as well because names of the feature types are QName. One old discussion about the topic is here http://osgeo-org.1560.x6.nabble.com/Re-Geoserver-devel-wms-getCapabilities-special-characters-td5160099.html.

Renaming the PostGIS columns is pretty easy to do with SQL View layer as Andrea told, or by creating the view into the database and publishing the view with Geoserver.

Special issue with Geoserver is that layers are usually automatically published as WMS and WFS or WCS with the same layer/featuretype/coverage name and while WMS and WCS 1.0.0 accepts anything as layer name WFS and WCS 2.0 are strict.

-Jukka Rahkonen-

Matthias Müller wrote:

Hi Andrea,

thanks for the quick reply.

Just for the record: Correct behaviour towards the client also matters in option (2). Possibilities for handling wrong GML are:

2a) return an HTTP error (fail early)
2b) hide invalid layers in Capabilities
2c) disable GML output on a particular WFS endpoint if one of the layers is invalid (contradicts WFS spec!)
2d) disable WFS endpoint until data becomes valid

All of these are far from ideal, so option (2) might be the least favorable solution.

Kind regards,
Matthias

Am 15.06.2016 um 11:14 schrieb Andrea Aime:

Hi Matthias,

I believe this is intended behavior, it’s not up to the data store code to police attribute names

and ids. E.g., if you were just using this layer for WMS, there would not be any problem,

serving also WFS is your choice, many do, but not everybody.

We have been over this a few times without any good consensus (and more importantly,

without any funding to take action on it), various positions are:

  • Do not allow publishing if the data is not compatible with all services

  • Warn the admin if a layer is not compatible with some service restrictions

  • Perform on the fly mappings to avoid this issue, with different mappings per service

The quickest solution for you is probably to setup a sql view that amends the names and

ids so that they are compliant. This works as long as you don’t have to edit via WFS-T of course.

Cheers

Andrea

On Wed, Jun 15, 2016 at 10:58 AM, Matthias Müller <matthias_mueller@anonymised.com4476…> wrote:

Hello everyone,

We are running a Geoserver instance with Postgres/PostGIS backend. Everything
seems to work fine so far, except for WFS with GML.

We have two specific issues here:

  1. The auto-generated feature IDs (see [1]) do not comply with the QName
    convention. We use UUIDs to identify our datasets and Geoserver seems to uses
    this UUID directly to generate Feature IDs. Since QNames must not start with a
    number, we get invalid GML for roughly 10 out of 16 datasets. Most
    Validators/Clients we have tested fail to parse this GML (QGIS among them).

  2. Geoserver uses Postgres column names as feature attribute names. Since
    Postgres column names are not restricted to QNames (e.g. “12345”, “km/s” are
    valid column names), we sometimes run into similar issues here.

Is this intended behavior, i.e. is the dataset provider responsible for
ensuring QName compliant attribute names, or is it a bug?

Kind regards,
Matthias

p.s.: Geoserver version is 2.8.3

[1]:

gml:featureMember
<ckandev:402b28fe-af35-463f-a78f-47f121cea236
fid=“402b28fe-af35-463f-a78f-47f121cea236.1”>
ckandev:X51.003874</ckandev:X>
ckandev:Y13.726043</ckandev:Y>
ckandev:Timestamp2012-07-20T:10:00GMT+1</ckandev:Timestamp>
ckandev:PlaceBannewitz</ckandev:Place>
ckandev:Zn_1000_-_400___micro_g_per_g_127.3936706</ckandev:Zn_1000_-400___micro_g_per_g>
ckandev:Zn_400_-_100___micro_g_per_g_418.9342066</ckandev:Zn_400_-100___micro_g_per_g>
ckandev:Zn_100_-_63___micro_g_per_g_743.5718278</ckandev:Zn_100_-63___micro_g_per_g>
ckandev:Zn_63_-_0.45___micro_g_per_g_528.2146161</ckandev:Zn_63_-0.45___micro_g_per_g>
ckandev:Zn_SUMM___micro_g_per_g_377.8245751</ckandev:Zn_SUMM___micro_g_per_g_>
ckandev:Cu_1000_-_400___micro_g_per_g_48.0749849</ckandev:Cu_1000_-400___micro_g_per_g>
ckandev:Cu_400_-_100___micro_g_per_g_112.1762429</ckandev:Cu_400_-100___micro_g_per_g>
ckandev:Cu_100_-_63___micro_g_per_g_164.8965903</ckandev:Cu_100_-63___micro_g_per_g>
ckandev:Cu_63_-_0.45___micro_g_per_g_151.0506145</ckandev:Cu_63_-0.45___micro_g_per_g>
ckandev:Cu_SUMM___micro_g_per_g_105.0860617</ckandev:Cu_SUMM___micro_g_per_g_>
ckandev:Cd_1000_-_400___micro_g_per_g_0.056641806</ckandev:Cd_1000_-400___micro_g_per_g>
ckandev:Cd_400_-_100___micro_g_per_g_0.166999107</ckandev:Cd_400_-100___micro_g_per_g>
ckandev:Cd_100_-_63___micro_g_per_g_0.264253773</ckandev:Cd_100_-63___micro_g_per_g>
ckandev:Cd_63_-_0.45___micro_g_per_g_0.278181578</ckandev:Cd_63_-0.45___micro_g_per_g>
ckandev:Cd_SUMM___micro_g_per_g_0.1663264</ckandev:Cd_SUMM___micro_g_per_g_>
ckandev:Shape
<gml:Point srsName=“http://www.opengis.net/gml/srs/epsg.xml#4326”>
<gml:coordinates xmlns:gml=“http://www.opengis.net/gml
decimal=“.” cs=“,” ts=" ">13.726043,51.003874</gml:coordinates>
</gml:Point>
</ckandev:Shape>
</ckandev:402b28fe-af35-463f-a78f-47f121cea236>
</gml:featureMember>


What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity planning
reports. http://pubads.g.doubleclick.net/gampad/clk?id=1444514421&iu=/41014381


Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

==

GeoServer Professional Services from the experts! Visit

http://goo.gl/it488V for more information.

==

Ing. Andrea Aime

@geowolf

Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)

phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.