[Geoserver-devel] GSIP proposal - add CITE data to normal data set?

GeoServer GeoServer Developer
1

After some work on CITE, I'm unhappy with the effort required to do a local
test (since you have to set up the test data). So I was going to suggest that
we modify the "normal" release build to include a working test environment
such that tests will pass against a routine build without needing to modify
the data directory.

So the first question was whether there are general +/- indications on whether
that is achievable technically / legally.

Assuming that is worthy of a GSIP, how I go about it? Based on
http://docs.geoserver.org/latest/en/developer/policies/gsip.html#implementing-a-gsip I tried to find the template or an example. It looks like
https://github.com/geoserver/geoserver.github.io/wiki/Proposals has no
content.

Any suggestions on the way forward here?

Brad

2

On Mon, Aug 24, 2015 at 11:28 AM, Brad Hards <bradh@anonymised.com> wrote:

After some work on CITE, I'm unhappy with the effort required to do a local
test (since you have to set up the test data). So I was going to suggest
that
we modify the "normal" release build to include a working test environment
such that tests will pass against a routine build without needing to modify
the data directory.

Interesting idea... with some gotchas.

The first one is that the CITE tests cannot be run against shapefiles due
to their attribute naming and data types
limitations... thus, we would have to ship H2 data stores with the right
data for the WFS compliance tests, instead,
and as a result, bundle the H2 store as a core one.
I'm not against, but would like to hear other devs opinions.

Another one is that the CITE tests expect their given data set, and I'm not
sure what happens
if other layers are made available as well.
We could try to use separate workspaces and use workspace specific
services, but WFS 1.0
needs two separate workspaces, so at least that one would have to be run
against global services.

Then there are the WCS tests, which fail if there is more than one version
of WCS deployed,
one has to selectively pull jars out of WEB-INF... again, we could add a
new feature to selectively
enable/disable WCS versions, would be nice, but to run the different tests,
one would still have
to unckeck the other versions... I guess I'd rather fix the CITE tests
instead.

For CSW tests, there is a gotcha that we would need to run against the
"basic store" instead
of the internal catalog one, so again a classpath manipulation. We could
make this configurable
again, or try to work on a composite store that allows more than one
backend store to be
used (would be useful too).

And then there are the other new tests that we are hopefully going to add,
not sure what
their requisites are, believe for example that WCS 2.0 has its own set of
expected data.

Hum... cannot think of other issues off the top of my head, but the ones
above are more than
enough imho :slight_smile:

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

3

On Mon, 24 Aug 2015 12:01:53 PM Andrea Aime wrote:

On Mon, Aug 24, 2015 at 11:28 AM, Brad Hards <bradh@anonymised.com> wrote:
> After some work on CITE, I'm unhappy with the effort required to do a
> local
> test (since you have to set up the test data). So I was going to suggest
> that
> we modify the "normal" release build to include a working test environment
> such that tests will pass against a routine build without needing to
> modify
> the data directory.

Interesting idea... with some gotchas.

The first one is that the CITE tests cannot be run against shapefiles due
to their attribute naming and data types
limitations... thus, we would have to ship H2 data stores with the right
data for the WFS compliance tests, instead,
and as a result, bundle the H2 store as a core one.
I'm not against, but would like to hear other devs opinions.

I was thinking perhaps property files might be an option.

Another one is that the CITE tests expect their given data set, and I'm not
sure what happens
if other layers are made available as well.

I think that would be a bug :slight_smile:

We could try to use separate workspaces and use workspace specific
services, but WFS 1.0
needs two separate workspaces, so at least that one would have to be run
against global services.

OK. Will investigate.

Then there are the WCS tests, which fail if there is more than one version
of WCS deployed,
one has to selectively pull jars out of WEB-INF... again, we could add a
new feature to selectively
enable/disable WCS versions, would be nice, but to run the different tests,
one would still have
to unckeck the other versions... I guess I'd rather fix the CITE tests
instead.

OK.

For CSW tests, there is a gotcha that we would need to run against the
"basic store" instead
of the internal catalog one, so again a classpath manipulation. We could
make this configurable
again, or try to work on a composite store that allows more than one
backend store to be
used (would be useful too).

OK.

And then there are the other new tests that we are hopefully going to add,
not sure what
their requisites are, believe for example that WCS 2.0 has its own set of
expected data.

Maybe a separate workspace could be an option here.

Hum... cannot think of other issues off the top of my head, but the ones
above are more than
enough imho :slight_smile:

Maybe we could have a progressive delivery on this. Say start with WMS 1.1.1
and 1.3, and work up?

Brad

4

On Mon, Aug 24, 2015 at 12:34 PM, Brad Hards <bradh@anonymised.com> wrote:

> The first one is that the CITE tests cannot be run against shapefiles due
> to their attribute naming and data types
> limitations... thus, we would have to ship H2 data stores with the right
> data for the WFS compliance tests, instead,
> and as a result, bundle the H2 store as a core one.
> I'm not against, but would like to hear other devs opinions.
I was thinking perhaps property files might be an option.

Hum... not sure, we should check the WFS 2.0 tests, property data stores
cannot do joins, only JDBC ones can.

> Another one is that the CITE tests expect their given data set, and I'm
not
> sure what happens
> if other layers are made available as well.
I think that would be a bug :slight_smile:

Questionable, as they give you a specific data set to be exposed, but
I believe that if the tests has issues, we could talk our way with OGC into
modifying the tests, if we are the ones providing the patches.

> And then there are the other new tests that we are hopefully going to
add,
> not sure what
> their requisites are, believe for example that WCS 2.0 has its own set of
> expected data.
Maybe a separate workspace could be an option here.

Yep.

> Hum... cannot think of other issues off the top of my head, but the ones
> above are more than
> enough imho :slight_smile:
Maybe we could have a progressive delivery on this. Say start with WMS
1.1.1
and 1.3, and work up?

Yeah, no objection here, maybe we could create a citewms workspace
where to put those data sets. It should be easy enough.

Let's hear how others feel about it. This would also eventually enable
to stand up a single GeoServer allowing to run all CITE tests, which
would be good as well.

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

5

Wonder if we could make an CITE store page backed by hsql with a reset button to reload the dataset.

···

On Mon, Aug 24, 2015 at 12:34 PM, Brad Hards <bradh@anonymised.com> wrote:

The first one is that the CITE tests cannot be run against shapefiles due
to their attribute naming and data types
limitations… thus, we would have to ship H2 data stores with the right
data for the WFS compliance tests, instead,
and as a result, bundle the H2 store as a core one.
I’m not against, but would like to hear other devs opinions.
I was thinking perhaps property files might be an option.

Hum… not sure, we should check the WFS 2.0 tests, property data stores
cannot do joins, only JDBC ones can.

Another one is that the CITE tests expect their given data set, and I’m not
sure what happens
if other layers are made available as well.
I think that would be a bug :slight_smile:

Questionable, as they give you a specific data set to be exposed, but
I believe that if the tests has issues, we could talk our way with OGC into
modifying the tests, if we are the ones providing the patches.

And then there are the other new tests that we are hopefully going to add,
not sure what
their requisites are, believe for example that WCS 2.0 has its own set of
expected data.
Maybe a separate workspace could be an option here.

Yep.

Hum… cannot think of other issues off the top of my head, but the ones
above are more than
enough imho :slight_smile:
Maybe we could have a progressive delivery on this. Say start with WMS 1.1.1
and 1.3, and work up?

Yeah, no objection here, maybe we could create a citewms workspace
where to put those data sets. It should be easy enough.

Let’s hear how others feel about it. This would also eventually enable
to stand up a single GeoServer allowing to run all CITE tests, which
would be good as well.

Cheers

Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

6

On Thu, Aug 27, 2015 at 6:14 PM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

Wonder if we could make an CITE store page backed by hsql with a reset
button to reload the dataset.

That's an excellent idea, since WFS tests with failures ruin the test data,
so you have to reload it
before the next test

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

7

Here is the feature request https://osgeo-org.atlassian.net/browse/GEOS-7178

···

On 27 August 2015 at 09:16, Andrea Aime <andrea.aime@anonymised.com> wrote:

–
Jody Garnett

On Thu, Aug 27, 2015 at 6:14 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Wonder if we could make an CITE store page backed by hsql with a reset button to reload the dataset.

That’s an excellent idea, since WFS tests with failures ruin the test data, so you have to reload it
before the next test

Cheers

Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.