[Geoserver-devel] Various INSPIRE extension changes (adding to WCS plus others)

I have been working on https://osgeo-org.atlassian.net/browse/GEOS-6435 as the INSPIRE maintenance group is moving towards including WCS as an option for download services. Along the way I came across a number of other changes that seemed worth making to the INSPIRE extension. The actual order I did these was all mixed up but I've tried to separate them into a reasonably coherent set of steps. I'd like some advice on whether to make one big pull request for all the changes or separate them into different parts as they aren't completely independent.

I have a working branch at https://github.com/bgsmase/geoserver/tree/temp_inspire_on_master_2 which contains a series of commits on top of master as follows:
1 - Point to the INSPIRE view schema at http://inspire.ec.europa.eu/schemas/inspire_vs/1.0/inspire_vs.xsd rather than copy locally hosted by GeoServer. (Should I file a ticket for this?)
2 - Miscellaneous tidying up of extended capabilities output including:
        Remove unnecessary namespace declarations
        Remove unnecessary xsi:type attributes
        Allow the service MetadataURL/MediaType element to be omitted. It is optional. I intend to add the full list of 20+ types enumerated in the INSPIRE schema later. Currently only two options are allowed.
        Don't include the extended capabilities element if the mandatory parameters have not been set.
        More tests for different combinations of parameters
        (Reluctant to make separate tickets/commits for all these)
3 - Add option to set the SpatialDataSetIdentifier/@metadataURL attribute. We would like to be able to do this (I could file a ticket for this specifically).
4 - Add INSPIRE extended capabilities to WCS [GEOS-6435]

Any comments / suggestions on any of the changes and, if they look OK, should I fold them up into one pull request or separate them along the items listed above?

Marcus Sen
British Geological Survey
Keyworth
Nottingham
NG12 5GG

Web: http://www.bgs.ac.uk

This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.

On Tue, May 19, 2015 at 5:37 PM, Sen, Marcus A. <mase@anonymised.com> wrote:

I have been working on https://osgeo-org.atlassian.net/browse/GEOS-6435
as the INSPIRE maintenance group is moving towards including WCS as an
option for download services. Along the way I came across a number of other
changes that seemed worth making to the INSPIRE extension. The actual order
I did these was all mixed up but I've tried to separate them into a
reasonably coherent set of steps. I'd like some advice on whether to make
one big pull request for all the changes or separate them into different
parts as they aren't completely independent.

I have a working branch at
https://github.com/bgsmase/geoserver/tree/temp_inspire_on_master_2 which
contains a series of commits on top of master as follows:
1 - Point to the INSPIRE view schema at
http://inspire.ec.europa.eu/schemas/inspire_vs/1.0/inspire_vs.xsd rather
than copy locally hosted by GeoServer. (Should I file a ticket for this?)

Normally we prefer to point to local hosted schemas, as opposed to remote
ones, because in some enviroments there is just
no internet wide access and this allows clients to still download the
schemas.

Also, OGC has the habit of changing the schemas over time, without changing
the file names, suddenly rendering schema
invalid something that was valid until the day before (which of course
makes it impossible to keep an already deployed production
instance up, the results become invalid overnight without anybody being
notified).

Now, inspire wise I guess we can rule out the restricted enviroments, but
I'm bot sure if INSPIRE is up to the same business as OGC
and changes the schemas in place, without creating a n+1 version (or
n.0.x+1 version).
In general, I'd leave the administration an option to point to the
canonical schemas online, or use the local ones (as we do in WFS
for example).

2 - Miscellaneous tidying up of extended capabilities output including:
        Remove unnecessary namespace declarations
        Remove unnecessary xsi:type attributes

These are un-necessary... why? Because we derive them by importing the DSL
schema? Or just because they are not used? (sorry to ask, I'm not the
original author of the inspire plugin)

        Allow the service MetadataURL/MediaType element to be omitted. It
is optional. I intend to add the full list of 20+ types enumerated in the
INSPIRE schema later. Currently only two options are allowed.
        Don't include the extended capabilities element if the mandatory
parameters have not been set.
        More tests for different combinations of parameters
        (Reluctant to make separate tickets/commits for all these)

I lost you there, the comment in the code says it's not used in download
services, but what about view ones, which we also support as WMS?

3 - Add option to set the SpatialDataSetIdentifier/@metadataURL attribute.
We would like to be able to do this (I could file a ticket for this
specifically).
4 - Add INSPIRE extended capabilities to WCS [GEOS-6435]

Any comments / suggestions on any of the changes and, if they look OK,
should I fold them up into one pull request or separate them along the
items listed above?

About the extra tickets, you might want to make a generic "Assorted inspire
plugin improvements" one, provided you give a detailed
list of what changed, for the release notes and changelogs.

Had a quick look at the patches, from a glance I don't see anything wrong,
but others will probably want to review
once you make a pull request.
About the pull request, please follow our norms, one commit per jira
ticket, with the ticket number and title as the commit message

Cheers
Andrea

--

Meet us at the INSPIRE Conference in Lisbon 25-29 May 2015! Visit
http://goo.gl/WHKDXT for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

-----Original Message-----
From: Andrea Aime [mailto:andrea.aime@…1268…]
Sent: 20 May 2015 15:34

...

Normally we prefer to point to local hosted schemas, as opposed to remote
ones, because in some enviroments there is just
no internet wide access and this allows clients to still download the schemas.

In this case the locally hosted Schema is a very short one that imports two other Schemas from remote locations so it doesn't help for an isolated server. Also the download service schema location is a remote one, not locally hosted. This wasn't my primary concern but I noticed a comment in the code asking whether the xsd still needed to be locally hosted so I thought I'd just go ahead and make the change.

...

      2 - Miscellaneous tidying up of extended capabilities output
including:
              Remove unnecessary namespace declarations
              Remove unnecessary xsi:type attributes

These are un-necessary... why? Because we derive them by importing the
DSL schema? Or just because they are not used? (sorry to ask, I'm not the
original author of the inspire plugin)

The namespace declarations aren't used. The xsi:type attributes aren't needed as the Schemas define the concrete type of the elements to which they were attached.

              Allow the service MetadataURL/MediaType element to be
omitted. It is optional. I intend to add the full list of 20+ types enumerated in
the INSPIRE schema later. Currently only two options are allowed.
              Don't include the extended capabilities element if the mandatory
parameters have not been set.
              More tests for different combinations of parameters
              (Reluctant to make separate tickets/commits for all these)

I lost you there, the comment in the code says it's not used in download
services, but what about view ones, which we also support as WMS?

I'm afraid I'm lost now :frowning: Which code comment are you referring to and what is not used in download services? I'm not sure to which bit of my message you are referring. I've written it slightly differently in the ticket I filed (GEOS-7042) so maybe that will be clearer?

...

About the extra tickets, you might want to make a generic "Assorted inspire
plugin improvements" one, provided you give a detailed
list of what changed, for the release notes and changelogs.

I've split it into 4 tickets now with one being the "assorted stuff" one:
https://osgeo-org.atlassian.net/browse/GEOS-7041
https://osgeo-org.atlassian.net/browse/GEOS-7042
https://osgeo-org.atlassian.net/browse/GEOS-7043
https://osgeo-org.atlassian.net/browse/GEOS-6435

Had a quick look at the patches, from a glance I don't see anything wrong, but
others will probably want to review
once you make a pull request.
About the pull request, please follow our norms, one commit per jira ticket,
with the ticket number and title as the commit message

As there are dependencies between the way I've currently written the code for each bit I've just submitted a pull request for the top ticket in the list above and will make further ones depending on whether that gets accepted/rejected/modified.

Marcus Sen

This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.

On 22/05/15 09:10, Sen, Marcus A. wrote:

From: Andrea Aime [mailto:andrea.aime@anonymised.com]
Normally we prefer to point to local hosted schemas, as opposed to remote
ones, because in some enviroments there is just
no internet wide access and this allows clients to still download the schemas.

In this case the locally hosted Schema is a very short one that imports two other Schemas from remote locations so it doesn't help for an isolated server. Also the download service schema location is a remote one, not locally hosted. This wasn't my primary concern but I noticed a comment in the code asking whether the xsd still needed to be locally hosted so I thought I'd just go ahead and make the change.

Marcus, in my view this is a very strong argument for changing the schemaLocation. I think that there is no benefit to retaining the existing behaviour and adding a UI switch would just be unnecessary complexity. Your proposed behaviour would permit clients to compare schemaLocation URL to detect the the INSPIRE schema. +1 for changing the schemaLocation to canonical, and I am quoting your text from GEOS-7041 below as it is even more compelling.

Kind regards,
Ben.

https://osgeo-org.atlassian.net/browse/GEOS-7041

******

Currently the INSPIRE plugin puts a schemaLocation which includes "http://inspire.ec.europa.eu/schemas/inspire_vs/1.0 http://localhost:8080/geoserver/www/inspire/inspire_vs.xsd&quot; for the INSPIRE view service namespace. However, this local schema is a very minimal wrapper which imports two other schemas from the Internet:

<import namespace="http://www.opengis.net/wms&quot; schemaLocation="http://schemas.opengis.net/wms/1.3.0/capabilities_1_3_0.xsd&quot;/&gt;&lt;import namespace="http://inspire.ec.europa.eu/schemas/common/1.0&quot; schemaLocation="http://inspire.ec.europa.eu/schemas/common/1.0/common\.xsd&quot;/&gt;

So there doesn't seem to be any benefit from hosting this small wrapper locally rather than using the canonical location at http://inspire.ec.europa.eu/schemas/inspire_vs/1.0/inspire_vs.xsd .

******

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <http://transient.nz/&gt;
New Zealand

On Thu, May 21, 2015 at 11:10 PM, Sen, Marcus A. <mase@anonymised.com> wrote:

> -----Original Message-----
> From: Andrea Aime [mailto:andrea.aime@anonymised.com]
> Sent: 20 May 2015 15:34
...
> Normally we prefer to point to local hosted schemas, as opposed to remote
> ones, because in some enviroments there is just
> no internet wide access and this allows clients to still download the
schemas.
In this case the locally hosted Schema is a very short one that imports
two other Schemas from remote locations so it doesn't help for an isolated
server. Also the download service schema location is a remote one, not
locally hosted. This wasn't my primary concern but I noticed a comment in
the code asking whether the xsd still needed to be locally hosted so I
thought I'd just go ahead and make the change.

Ok, better not have it at all, than having it only half baked I guess.

...
> 2 - Miscellaneous tidying up of extended capabilities output
> including:
> Remove unnecessary namespace declarations
> Remove unnecessary xsi:type attributes
>
> These are un-necessary... why? Because we derive them by importing the
> DSL schema? Or just because they are not used? (sorry to ask, I'm not the
> original author of the inspire plugin)
The namespace declarations aren't used. The xsi:type attributes aren't
needed as the Schemas define the concrete type of the elements to which
they were attached.

> Allow the service MetadataURL/MediaType element to be
> omitted. It is optional. I intend to add the full list of 20+ types
enumerated in
> the INSPIRE schema later. Currently only two options are allowed.
> Don't include the extended capabilities element if the
mandatory
> parameters have not been set.
> More tests for different combinations of parameters
> (Reluctant to make separate tickets/commits for all these)
>
> I lost you there, the comment in the code says it's not used in download
> services, but what about view ones, which we also support as WMS?
I'm afraid I'm lost now :frowning: Which code comment are you referring to and
what is not used in download services? I'm not sure to which bit of my
message you are referring. I've written it slightly differently in the
ticket I filed (GEOS-7042) so maybe that will be clearer?

I meant this one, but it seems I read the diff backwards, so ignore it:
https://github.com/bgsmase/geoserver/commit/5610fd60981cd518084d5a3f91b9e6b44053ffdd#diff-49482e052bcb0002ddfb0bec680d3e8bL31

Cheers
Andrea

--

Meet us at the INSPIRE Conference in Lisbon 25-29 May 2015! Visit
http://goo.gl/WHKDXT for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------