[Geoserver-devel] More about live testint

GeoServer GeoServer Developer
1

Hi,
I'm going forward on the live testing thing and I'm
faced with a decision I'd like to discuss.

So far I've created a LiveData class for testing against
a data directory using only files as data stores, and
a LiveDbmsData that handles the following:
* grabbing a property file with connection params to the local db
   to be used for testing
* filtering catalog.xml with the the params contained in the fixture
   so that geoserver will connect to the db
* running a sql script file using the connection provided by
   a jdbc data store.

Now, there are a few issues.

The first is with the property file.
I've setup things so that the keys in the property file are
used everywhere: they have to be the name of the connection
params to the jdbc datastore used to run the script
(which will be looked up with DatastoreFinder), and they are
the tokens that will be replaced in catalog.xml.
It makes things simpler, no aliases to work with, thought it basically makes it impossible to setup two connections to two
different jdbc datastores. Do you see an isue with that?
It suits my needs and keeps things simple, but I'm open
to alternate suggestions.

The second is.. with the property file! :slight_smile:
We agreed that it would be good to lookup it into the
current user's home. Ok, let's say we look up in
.gstest/fixture.properties.
What if the file is not there? We do not run the test?
Or we copy a prototype file from the classpath and try
a connection test?
Of course if the connection test fails the test will
be disabled... I'll have to basically use the same code
as in GeoTool's OnlineTest... but I cannot use it
directly, since the run and check once mechanism
will have to be placed in a subclass of GeoServerAbstractTestSupport.

Suggestions on that one? I just copy some of the code from
OnlineTestCase to a GeoServerDbmsTestSupport? Any other
idea you'd like better?

Cheers
Andrea

2

Andrea Aime wrote:

Suggestions on that one? I just copy some of the code from
OnlineTestCase to a GeoServerDbmsTestSupport? Any other
idea you'd like better?
  

I have become more and more comfortable with maven profiles; I would rather specify the connection
data in a profile than have it located in my home directory.

Jody

3

The first is with the property file.
I've setup things so that the keys in the property file are
used everywhere: they have to be the name of the connection
params to the jdbc datastore used to run the script
(which will be looked up with DatastoreFinder), and they are
the tokens that will be replaced in catalog.xml.
It makes things simpler, no aliases to work with, thought it basically makes it impossible to setup two connections to two
different jdbc datastores. Do you see an isue with that?
It suits my needs and keeps things simple, but I'm open
to alternate suggestions.

For the purposes of testing i think it reasonable to limit to one type of database. I cant think of any use cases for otherwise. So +1.

The second is.. with the property file! :slight_smile:
We agreed that it would be good to lookup it into the
current user's home. Ok, let's say we look up in
.gstest/fixture.properties.
What if the file is not there? We do not run the test?
Or we copy a prototype file from the classpath and try
a connection test?
Of course if the connection test fails the test will
be disabled... I'll have to basically use the same code
as in GeoTool's OnlineTest... but I cannot use it
directly, since the run and check once mechanism
will have to be placed in a subclass of GeoServerAbstractTestSupport.

Suggestions on that one? I just copy some of the code from
OnlineTestCase to a GeoServerDbmsTestSupport? Any other
idea you'd like better?

I would say if the test fixture file does not exist, it does not run. I am thinking of it from a build server point of view and i simply just want to remove the file from the home directory and be done with it.

However I do think it is nice if the tests also handle the case of the database or external server being down gracefully.

Cheers
Andrea

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4007,47d03ff6281911804284693!

--
Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com

4

I have become more and more comfortable with maven profiles; I would rather specify the connection
data in a profile than have it located in my home directory.

I think this is a misuse of a profile imho. Profiles can too easily trip over one another, especially in multi module projects like geotools.

What i like about the existance of a properties file in the home directory is that you don't have to modify any "versioned" files. Modifying versioned files on the build server (where theoretically these tests should be run most of the time) is a nightmare as soon as someone commits that file, you get a slough of build failures.

Jody

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4007,47d0409d284802458217002!

--
Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com

5

I was more thinking of a commercial project I worked on; the connection information for the database in your workgroup was placed in settings xml and activated with a profile (that I had turned on by default). The nice thing was you could place a profiles.xml file next to your root pom.xml and accomplish different settings for different versions of the software without distrupting the committed pom.xml.

Cheers,
Jody

I have become more and more comfortable with maven profiles; I would rather specify the connection
data in a profile than have it located in my home directory.

I think this is a misuse of a profile imho. Profiles can too easily trip over one another, especially in multi module projects like geotools.

What i like about the existance of a properties file in the home directory is that you don't have to modify any "versioned" files. Modifying versioned files on the build server (where theoretically these tests should be run most of the time) is a nightmare as soon as someone commits that file, you get a slough of build failures.

Jody

-------------------------------------------------------------------------

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4007,47d0409d284802458217002!

6

Jody Garnett ha scritto:

I was more thinking of a commercial project I worked on; the connection information for the database in your workgroup was placed in settings xml and activated with a profile (that I had turned on by default). The nice thing was you could place a profiles.xml file next to your root pom.xml and accomplish different settings for different versions of the software without distrupting the committed pom.xml.

Hum, this would work fine for a single db connection, but what if
in the same build we need to deal with multiple databases (that is,
different tests in the same build use different database connections)?

With a property file I can point to a differently named one,
and the parameters inside of it can still follow the datastore
naming 1-1, that is, be "host","port","user",...

If we used a profile, to test different db we'd need to activate
multiple of them, meaning I would have to give a different name to
the params... a prefix based convention could be established, in
order to have params named wfsv.host or oracle.host and so on...

I see the following advantages/disadvantages:

profile:
* params can be set without forcing online tests to run, you
   can decide what to run and what not to run.
* on the flip side, to run a certain set of online builds, you
   have to forcefully name each profile on the command line
   (automatic profile activation is evil)

property:
* you set the property file in your home, if it's the and
   the connection succeeds, the test will run, otherwise
   not. So it's file driven. As a consequence, there is
   no easy way to disable a test on occasion, you have
   to remove/rename the associated property file

The property file limitation of automatic enabling could
be circumvented by looking for a system variable.
If the fixture file is named wfsv, we could make it so that the test
is skipped if the gs.wfsv=false is set.
In fact that could be the way to disable a test for multiple
runs as well: picture the situation in which multiple functional
tests are using the same connection (this will be the case
for wfsv). If the first test runs and it finds that the
connection is not possible, it logs a warning, set the
system variable, and none of the other tests will attempt
the connection at all (provided everything is run in the
same VM, which is the case for geoserver afaik, surefire
forking policy is set to "once").

I'll be going down the property file road, anyways, if the
PSC feels a more appropriate solution is needed, I'll
modify the code accordingly (I'll try to keep the param
lookup and usage isolated so that it's easy to change).

Cheers
Andrea

7

If we used a profile, to test different db we'd need to activate
multiple of them, meaning I would have to give a different name to
the params... a prefix based convention could be established, in
order to have params named wfsv.host or oracle.host and so on...

Originally I envision each test having a fixture. So if a test is named "VerioningTest", the test would look for a fixture file: "~/VersioningTest.properties". However after more thought i can see how this will be tedious to manage a bunch of difference fixtures which contain all the same info.

So... will there be some sort of method like "getFixture()" that returns the name? If my memory serves me correctly this is how we did it in gt2.

I see the following advantages/disadvantages:

profile:
* params can be set without forcing online tests to run, you
  can decide what to run and what not to run.
* on the flip side, to run a certain set of online builds, you
  have to forcefully name each profile on the command line
  (automatic profile activation is evil)

property:
* you set the property file in your home, if it's the and
  the connection succeeds, the test will run, otherwise
  not. So it's file driven. As a consequence, there is
  no easy way to disable a test on occasion, you have
  to remove/rename the associated property file

The property file limitation of automatic enabling could
be circumvented by looking for a system variable.
If the fixture file is named wfsv, we could make it so that the test
is skipped if the gs.wfsv=false is set.
In fact that could be the way to disable a test for multiple
runs as well: picture the situation in which multiple functional
tests are using the same connection (this will be the case
for wfsv). If the first test runs and it finds that the
connection is not possible, it logs a warning, set the
system variable, and none of the other tests will attempt
the connection at all (provided everything is run in the
same VM, which is the case for geoserver afaik, surefire
forking policy is set to "once").

I'll be going down the property file road, anyways, if the
PSC feels a more appropriate solution is needed, I'll
modify the code accordingly (I'll try to keep the param
lookup and usage isolated so that it's easy to change).

Cheers
Andrea

!DSPAM:4007,47d1119f325781137850744!

--
Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com

8

Justin Deoliveira wrote:

If we used a profile, to test different db we'd need to activate
multiple of them, meaning I would have to give a different name to
the params... a prefix based convention could be established, in
order to have params named wfsv.host or oracle.host and so on...
    

Originally I envision each test having a fixture. So if a test is named "VerioningTest", the test would look for a fixture file: "~/VersioningTest.properties". However after more thought i can see how this will be tedious to manage a bunch of difference fixtures which contain all the same info.
  

It seems like we could have a system for this that would be fairly manageable. We could make it so that the fixtures are resolved by looking for the fully qualified class name of the test class, like so:
org.geoserver.wfsv.VersioningTest.properties

And then if said properties file doesn't exist, we look at the package, then the parent package, etc. Then, if a package wishes to provide a fixture common to all tests in that package, then there'd be a properties file for the package instead of each class:
org.geoserver.wfsv.properties

And packages for which the defaults were fine would just not do anything, and they'd load a fixture from
org.geoserver.properties

I imagine that we'd want to have more specific properties files completely override less specific ones, but other than that we'd basically have inheritance for test fixtures.

-David

So... will there be some sort of method like "getFixture()" that returns the name? If my memory serves me correctly this is how we did it in gt2.

I see the following advantages/disadvantages:

profile:
* params can be set without forcing online tests to run, you
  can decide what to run and what not to run.
* on the flip side, to run a certain set of online builds, you
  have to forcefully name each profile on the command line
  (automatic profile activation is evil)

property:
* you set the property file in your home, if it's the and
  the connection succeeds, the test will run, otherwise
  not. So it's file driven. As a consequence, there is
  no easy way to disable a test on occasion, you have
  to remove/rename the associated property file

The property file limitation of automatic enabling could
be circumvented by looking for a system variable.
If the fixture file is named wfsv, we could make it so that the test
is skipped if the gs.wfsv=false is set.
In fact that could be the way to disable a test for multiple
runs as well: picture the situation in which multiple functional
tests are using the same connection (this will be the case
for wfsv). If the first test runs and it finds that the
connection is not possible, it logs a warning, set the
system variable, and none of the other tests will attempt
the connection at all (provided everything is run in the
same VM, which is the case for geoserver afaik, surefire
forking policy is set to "once").

I'll be going down the property file road, anyways, if the
PSC feels a more appropriate solution is needed, I'll
modify the code accordingly (I'll try to keep the param
lookup and usage isolated so that it's easy to change).

Cheers
Andrea

9

Justin Deoliveira ha scritto:

If we used a profile, to test different db we'd need to activate
multiple of them, meaning I would have to give a different name to
the params... a prefix based convention could be established, in
order to have params named wfsv.host or oracle.host and so on...

Originally I envision each test having a fixture. So if a test is named "VerioningTest", the test would look for a fixture file: "~/VersioningTest.properties". However after more thought i can see how this will be tedious to manage a bunch of difference fixtures which contain all the same info.

So... will there be some sort of method like "getFixture()" that returns the name? If my memory serves me correctly this is how we did it in gt2.

Yes but in GeoServer the testing setup is done in the data directory
setup class, which is not a test case. I made up things so
that a dbms oriented data directory setup class received the
fixture id.

It could be the class name, but that would make it hard to share
the same testing config between tests. Having a significant identifier
seems to me less complex and just as effective than using the
class or the package name... what am I missing?

Cheers
Andrea

PS: I'm about to commit the test classes as I have developed them,
then I'll write a mail to guide you in the review.

10

It seems like we could have a system for this that would be fairly manageable. We could make it so that the fixtures are resolved by looking for the fully qualified class name of the test class, like so:
org.geoserver.wfsv.VersioningTest.properties

And then if said properties file doesn't exist, we look at the package, then the parent package, etc. Then, if a package wishes to provide a fixture common to all tests in that package, then there'd be a properties file for the package instead of each class:
org.geoserver.wfsv.properties

And packages for which the defaults were fine would just not do anything, and they'd load a fixture from
org.geoserver.properties

I imagine that we'd want to have more specific properties files completely override less specific ones, but other than that we'd basically have inheritance for test fixtures.

-David

Yeah, I actually thought about pretty much the exact same scheme when i cant up for it before. Then it seemed to get a bit complicated and overkill for something as simple as an online test... so i opted out. But definitely could be a promising idea.

    
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4007,47d164f3122462092453641!

--
Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com