[Geoserver-devel] status of cite tests on trunk

GeoServer GeoServer Developer
1

Hi folks,

I spent today trying to get the cite tests back in shape. Here is where things are at.

  • wfs-1.0.0

There were a number of DescribeFeatureType test failures introduced by some behaviour that was brought in by WFS 2.0. Basically, and I need to check this again with the wfs 2.0 spec, the spec mandates that in a case where a DescribeFeatureType or GetFeature includes types from different namespaces that the schema should still declare a target namespace, the namespace of the first feature type. Anyways, this behaviour causes a validation failure in the wfs 1.0 schema validator.

Disabling this restores the cite tests. I am going to look into the wfs 2.0 schema and verify this requirement. However there is a wfs 2.0 unit test that fails with this behaviour taken out.

  • wfs-1.1.0

This one is a sad story. Unfortunately the wfs 1.1 cite tests (at least the version we currently run) are not meant to be run in the context of wfs 2.0. They basically have the same problem that the wcs 1.0 cite tests have in that the wcs 1.0 tests can’t pass if the wcs 1.1 service is active. Most of the failures are due to the assumption that 1.1 is highest version of the service available. There are a few other failures as well, but they seem fairly fixable.

So unfortunately the version issues leave us in a tricky spot. The solution for the problem on the wcs side is simply to remove wcs 1.1 while running the wcs 1.0 tests. Which is easy since it is its own module. But wfs doesn’t have this same luxury.

It could be that more recent versions of the tests don’t have these issues but unfortunately updating to the latest version of them is not trivial. Unfortunately geoserver doesn’t even pass the sanity checks any longer due to changes in the wfs 1.1 schema. Funny how software can be compliant one day, not change anything, and the next day be non-compliant.

So as i see it we have three options.

  1. We disable the tests that make assumptions about the highest service version. All in all this about a 15-20 tests out of about 500.

  2. We attempt to factor out wfs 2.0 into a separate module and disable it during the wfs 1.1 tests, the same solution we apply for wcs. For the most part this would probably just amount to factoring out just the bean definitions, but not sure if it would go any further than that. The minute we have to start moving code out of the main wfs module it becomes a pretty sizeable and ugly task.

  3. We bite the bullet and upgrade to the latest version of the cite tests and work with the cite team to deal with these issues, ideally getting patches submitted for fixing stuff like this. Obviously long term this is the most sustainable and robust solution but it also means we won’t be passing wfs 1.1 cite any time soon. It is also worth noting that the wfs 2.0 cite tests are going to be developed soon so eventually we are going to have to run a newer version of the tests if we want to be 2.0 compliant.

All things considered I think for now we should just apply solution (1). But at the same time start gradually working toward (3). WHich actually should be coming up as a priority for us at OpenGeo relatively soon as we have had some clients asking about cite compliance.

  • wms 1.3

The failures here are straight forward, just due to some mime types not matching output format names. Should be straight forward fix.

That’s it that’s all.

-Justin

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

2

Is it possible to create an additional WFS service URL that transparently set service=WFS and version=1.1.0 to requests, as a wrapper around the existing service? Would this allow GeoServer to masquerade as a WFS-1.1.0-only service?

If all the failures can be confirmed as being caused by CITE assumptions that 1.1.0 is the highest version available, then in my view we can take option (1) and claim CITE compliance with this qualification well documented.

On 11/05/12 12:17, Justin Deoliveira wrote:

Hi folks,

I spent today trying to get the cite tests back in shape. Here is where things are at.

* wfs-1.0.0

There were a number of DescribeFeatureType test failures introduced by some behaviour that was brought in by WFS 2.0. Basically, and I need to check this again with the wfs 2.0 spec, the spec mandates that in a case where a DescribeFeatureType or GetFeature includes types from different namespaces that the schema should still declare a target namespace, the namespace of the first feature type. Anyways, this behaviour causes a validation failure in the wfs 1.0 schema validator.

Disabling this restores the cite tests. I am going to look into the wfs 2.0 schema and verify this requirement. However there is a wfs 2.0 unit test that fails with this behaviour taken out.

* wfs-1.1.0

This one is a sad story. Unfortunately the wfs 1.1 cite tests (at least the version we currently run) are not meant to be run in the context of wfs 2.0. They basically have the same problem that the wcs 1.0 cite tests have in that the wcs 1.0 tests can't pass if the wcs 1.1 service is active. Most of the failures are due to the assumption that 1.1 is highest version of the service available. There are a few other failures as well, but they seem fairly fixable.

So unfortunately the version issues leave us in a tricky spot. The solution for the problem on the wcs side is simply to remove wcs 1.1 while running the wcs 1.0 tests. Which is easy since it is its own module. But wfs doesn't have this same luxury.

It could be that more recent versions of the tests don't have these issues but unfortunately updating to the latest version of them is not trivial. Unfortunately geoserver doesn't even pass the sanity checks any longer due to changes in the wfs 1.1 schema. Funny how software can be compliant one day, not change anything, and the next day be non-compliant.

So as i see it we have three options.

1. We disable the tests that make assumptions about the highest service version. All in all this about a 15-20 tests out of about 500.

2. We attempt to factor out wfs 2.0 into a separate module and disable it during the wfs 1.1 tests, the same solution we apply for wcs. For the most part this would probably just amount to factoring out just the bean definitions, but not sure if it would go any further than that. The minute we have to start moving code out of the main wfs module it becomes a pretty sizeable and ugly task.

3. We bite the bullet and upgrade to the latest version of the cite tests and work with the cite team to deal with these issues, ideally getting patches submitted for fixing stuff like this. Obviously long term this is the most sustainable and robust solution but it also means we won't be passing wfs 1.1 cite any time soon. It is also worth noting that the wfs 2.0 cite tests are going to be developed soon so eventually we are going to have to run a newer version of the tests if we want to be 2.0 compliant.

All things considered I think for now we should just apply solution (1). But at the same time start gradually working toward (3). WHich actually should be coming up as a priority for us at OpenGeo relatively soon as we have had some clients asking about cite compliance.

* wms 1.3

The failures here are straight forward, just due to some mime types not matching output format names. Should be straight forward fix.

That's it that's all.

-Justin

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

3

On Fri, May 11, 2012 at 6:17 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

It could be that more recent versions of the tests don't have these issues
but unfortunately updating to the latest version of them is not trivial.
Unfortunately geoserver doesn't even pass the sanity checks any longer due
to changes in the wfs 1.1 schema. Funny how software can be compliant one
day, not change anything, and the next day be non-compliant.

Ah, living la vida loca...

So as i see it we have three options.

1. We disable the tests that make assumptions about the highest service
version. All in all this about a 15-20 tests out of about 500.

2. We attempt to factor out wfs 2.0 into a separate module and disable it
during the wfs 1.1 tests, the same solution we apply for wcs. For the most
part this would probably just amount to factoring out just the bean
definitions, but not sure if it would go any further than that. The minute
we have to start moving code out of the main wfs module it becomes a
pretty sizeable and ugly task.

3. We bite the bullet and upgrade to the latest version of the cite tests
and work with the cite team to deal with these issues, ideally getting
patches submitted for fixing stuff like this. Obviously long term this is
the most sustainable and robust solution but it also means we won't be
passing wfs 1.1 cite any time soon. It is also worth noting that the wfs 2.0
cite tests are going to be developed soon so eventually we are going to have
to run a newer version of the tests if we want to be 2.0 compliant.

May I suggest also

4. Add to services a checkbox list that allows to selectively
enable/disable a certain
    service version

The above might be a welcome new feature from the users p.o.v. and should
help solving our version dance leaving only one active in the data dir
for its cits test. Not sure if it's more or less work than 1), the check is just
another dispatcher callback, the config/GUI as usual is the annoying bit.

All things considered I think for now we should just apply solution (1). But
at the same time start gradually working toward (3). WHich actually should
be coming up as a priority for us at OpenGeo relatively soon as we have had
some clients asking about cite compliance.

(3) looks good to me as well of course.

* wms 1.3

The failures here are straight forward, just due to some mime types not
matching output format names. Should be straight forward fix.

Hopefully, I'll have a look into it :slight_smile:

Cheers
Andrea

--
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

4

On Fri, May 11, 2012 at 3:06 AM, Andrea Aime
<andrea.aime@anonymised.com> wrote:

On Fri, May 11, 2012 at 6:17 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

It could be that more recent versions of the tests don't have these issues
but unfortunately updating to the latest version of them is not trivial.
Unfortunately geoserver doesn't even pass the sanity checks any longer due
to changes in the wfs 1.1 schema. Funny how software can be compliant one
day, not change anything, and the next day be non-compliant.

Ah, living la vida loca...

So as i see it we have three options.

1. We disable the tests that make assumptions about the highest service
version. All in all this about a 15-20 tests out of about 500.

2. We attempt to factor out wfs 2.0 into a separate module and disable it
during the wfs 1.1 tests, the same solution we apply for wcs. For the most
part this would probably just amount to factoring out just the bean
definitions, but not sure if it would go any further than that. The minute
we have to start moving code out of the main wfs module it becomes a
pretty sizeable and ugly task.

3. We bite the bullet and upgrade to the latest version of the cite tests
and work with the cite team to deal with these issues, ideally getting
patches submitted for fixing stuff like this. Obviously long term this is
the most sustainable and robust solution but it also means we won't be
passing wfs 1.1 cite any time soon. It is also worth noting that the wfs 2.0
cite tests are going to be developed soon so eventually we are going to have
to run a newer version of the tests if we want to be 2.0 compliant.

May I suggest also

4. Add to services a checkbox list that allows to selectively
enable/disable a certain
service version

The above might be a welcome new feature from the users p.o.v. and should
help solving our version dance leaving only one active in the data dir
for its cits test. Not sure if it's more or less work than 1), the check is just
another dispatcher callback, the config/GUI as usual is the annoying bit.

Sounds like a really good thing to me. +1

All things considered I think for now we should just apply solution (1). But
at the same time start gradually working toward (3). WHich actually should
be coming up as a priority for us at OpenGeo relatively soon as we have had
some clients asking about cite compliance.

(3) looks good to me as well of course.

* wms 1.3

The failures here are straight forward, just due to some mime types not
matching output format names. Should be straight forward fix.

Deja vu. I wanted to "fix" that a while back but people were too used
to our aliases (like format=OpenLayers, etc). I guess what we could do
is to refrain from advertising mime type aliases when the strict CITE
compliance flag is set?

Cheers,
Gabriel

Hopefully, I'll have a look into it :slight_smile:

Cheers
Andrea

--
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.

5

On Thu, May 10, 2012 at 10:53 PM, Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com> wrote:

Is it possible to create an additional WFS service URL that transparently set service=WFS and version=1.1.0 to requests, as a wrapper around the existing service? Would this allow GeoServer to masquerade as a WFS-1.1.0-only service?

Yeah, I thought about this as well. Changing the urls in the capabilities document to literally force a 1.1.0 even if the client doesn’t intend on sending them. Sort of weary of hacking the capabilities document.

If all the failures can be confirmed as being caused by CITE assumptions that 1.1.0 is the highest version available, then in my view we can take option (1) and claim CITE compliance with this qualification well documented.

Unfortunately we can’t claim cite compliance since the OGC has been changing the test suite, and as far as i know there is no versioning of compliance. So literally at one point we passed these exact same tests and were considered compliant but today we are not.

On 11/05/12 12:17, Justin Deoliveira wrote:

Hi folks,

I spent today trying to get the cite tests back in shape. Here is where things are at.

  • wfs-1.0.0

There were a number of DescribeFeatureType test failures introduced by some behaviour that was brought in by WFS 2.0. Basically, and I need to check this again with the wfs 2.0 spec, the spec mandates that in a case where a DescribeFeatureType or GetFeature includes types from different namespaces that the schema should still declare a target namespace, the namespace of the first feature type. Anyways, this behaviour causes a validation failure in the wfs 1.0 schema validator.

Disabling this restores the cite tests. I am going to look into the wfs 2.0 schema and verify this requirement. However there is a wfs 2.0 unit test that fails with this behaviour taken out.

  • wfs-1.1.0

This one is a sad story. Unfortunately the wfs 1.1 cite tests (at least the version we currently run) are not meant to be run in the context of wfs 2.0. They basically have the same problem that the wcs 1.0 cite tests have in that the wcs 1.0 tests can’t pass if the wcs 1.1 service is active. Most of the failures are due to the assumption that 1.1 is highest version of the service available. There are a few other failures as well, but they seem fairly fixable.

So unfortunately the version issues leave us in a tricky spot. The solution for the problem on the wcs side is simply to remove wcs 1.1 while running the wcs 1.0 tests. Which is easy since it is its own module. But wfs doesn’t have this same luxury.

It could be that more recent versions of the tests don’t have these issues but unfortunately updating to the latest version of them is not trivial. Unfortunately geoserver doesn’t even pass the sanity checks any longer due to changes in the wfs 1.1 schema. Funny how software can be compliant one day, not change anything, and the next day be non-compliant.

So as i see it we have three options.

  1. We disable the tests that make assumptions about the highest service version. All in all this about a 15-20 tests out of about 500.

  2. We attempt to factor out wfs 2.0 into a separate module and disable it during the wfs 1.1 tests, the same solution we apply for wcs. For the most part this would probably just amount to factoring out just the bean definitions, but not sure if it would go any further than that. The minute we have to start moving code out of the main wfs module it becomes a pretty sizeable and ugly task.

  3. We bite the bullet and upgrade to the latest version of the cite tests and work with the cite team to deal with these issues, ideally getting patches submitted for fixing stuff like this. Obviously long term this is the most sustainable and robust solution but it also means we won’t be passing wfs 1.1 cite any time soon. It is also worth noting that the wfs 2.0 cite tests are going to be developed soon so eventually we are going to have to run a newer version of the tests if we want to be 2.0 compliant.

All things considered I think for now we should just apply solution (1). But at the same time start gradually working toward (3). WHich actually should be coming up as a priority for us at OpenGeo relatively soon as we have had some clients asking about cite compliance.

  • wms 1.3

The failures here are straight forward, just due to some mime types not matching output format names. Should be straight forward fix.

That’s it that’s all.

-Justin

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

–
Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com
Software Engineer
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

6

On Fri, May 11, 2012 at 4:23 PM, Gabriel Roldan <groldan@anonymised.com> wrote:

* wms 1.3

The failures here are straight forward, just due to some mime types not
matching output format names. Should be straight forward fix.

Deja vu. I wanted to "fix" that a while back but people were too used
to our aliases (like format=OpenLayers, etc). I guess what we could do
is to refrain from advertising mime type aliases when the strict CITE
compliance flag is set?

Sounds like a good idea to me indeed

Cheers
Andrea

--
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

7

On Fri, May 11, 2012 at 8:23 AM, Gabriel Roldan <groldan@anonymised.com> wrote:

On Fri, May 11, 2012 at 3:06 AM, Andrea Aime
<andrea.aime@anonymised.com> wrote:

On Fri, May 11, 2012 at 6:17 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

It could be that more recent versions of the tests don’t have these issues
but unfortunately updating to the latest version of them is not trivial.
Unfortunately geoserver doesn’t even pass the sanity checks any longer due
to changes in the wfs 1.1 schema. Funny how software can be compliant one
day, not change anything, and the next day be non-compliant.

Ah, living la vida loca…

So as i see it we have three options.

  1. We disable the tests that make assumptions about the highest service
    version. All in all this about a 15-20 tests out of about 500.

  2. We attempt to factor out wfs 2.0 into a separate module and disable it
    during the wfs 1.1 tests, the same solution we apply for wcs. For the most
    part this would probably just amount to factoring out just the bean
    definitions, but not sure if it would go any further than that. The minute
    we have to start moving code out of the main wfs module it becomes a
    pretty sizeable and ugly task.

  3. We bite the bullet and upgrade to the latest version of the cite tests
    and work with the cite team to deal with these issues, ideally getting
    patches submitted for fixing stuff like this. Obviously long term this is
    the most sustainable and robust solution but it also means we won’t be
    passing wfs 1.1 cite any time soon. It is also worth noting that the wfs 2.0
    cite tests are going to be developed soon so eventually we are going to have
    to run a newer version of the tests if we want to be 2.0 compliant.

May I suggest also

  1. Add to services a checkbox list that allows to selectively
    enable/disable a certain
    service version

The above might be a welcome new feature from the users p.o.v. and should
help solving our version dance leaving only one active in the data dir
for its cits test. Not sure if it’s more or less work than 1), the check is just
another dispatcher callback, the config/GUI as usual is the annoying bit.

Sounds like a really good thing to me. +1

Yeah, i like this solution as well and will look into it. I am not sure it is just another dispatcher callback though since in this case what I really want is to do is filter down ServiceInfo objects based on what versions are enabled, and not simply throw back an exception when a user tries to use a disabled version. And then there is logic outside the dispatcher as well mostly in the GetCapabilities operation doing additional version negotiation. And finally yes, it involves a config change. All in all this might not be a change suitable for 2.2 as we move toward stability. I could be wrong though.

All things considered I think for now we should just apply solution (1). But
at the same time start gradually working toward (3). WHich actually should
be coming up as a priority for us at OpenGeo relatively soon as we have had
some clients asking about cite compliance.

(3) looks good to me as well of course.

  • wms 1.3

The failures here are straight forward, just due to some mime types not
matching output format names. Should be straight forward fix.

Deja vu. I wanted to “fix” that a while back but people were too used
to our aliases (like format=OpenLayers, etc). I guess what we could do
is to refrain from advertising mime type aliases when the strict CITE
compliance flag is set?

Cheers,
Gabriel

Hopefully, I’ll have a look into it :slight_smile:

Cheers
Andrea

–
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.