[Geoserver-devel] Http Unit Tests

GeoServer GeoServer Developer
1

Hi all,

I have spent some time playing with HttpUnit (http://httpunit.sourceforge.net/) and found it to be a very effective means of testing.

Unit tests only get us so far (we dont have many) as they test things in isolation. Http unit tests are much like CITE tests in that they are a good tool for integration testing.

I added the httpunit library to the codebase. There are also some example tests under the httptest directory. I wanted to seperate them from the normal unit tests since they require a running instance of geoserver.

I also wrote some docs on how to execute the tests.

http://docs.codehaus.org/display/GEOSDOC/Project+Conventions

Under Testing.

Please check them out and make any comments/corrections.

Thanks.

Justin

p.s. If someone was really keen they could write an http unit test which would churn through all the files in the demo directory.

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org

2

Justin Deoliveira wrote:

Hi all,

Hi

Excellent work. I have started to do some post vs. get testing for the
GetMap requests.

I have some questions in that regard:

1. For a test setup is there a map that I can use that I know will be
there? Like states. Or would I have to upload a map in my test suite?

2. I also find it a bit odd that the httptest.properties is commited to
subversion. Each of the developers could have their own setup, e.g.
different port number. Since it is in the code repository, someone are
in danger of committing it making the life of others harder.

My suggestion would be to e.g. add a httptest.properties.example file,
and document that testers must copy that file into httptest.properties
using their own setup parameters.

Magne

I have spent some time playing with HttpUnit
(http://httpunit.sourceforge.net/) and found it to be a very effective
means of testing.

Unit tests only get us so far (we dont have many) as they test things in
isolation. Http unit tests are much like CITE tests in that they are a
good tool for integration testing.

I added the httpunit library to the codebase. There are also some
example tests under the httptest directory. I wanted to seperate them
from the normal unit tests since they require a running instance of
geoserver.

I also wrote some docs on how to execute the tests.

http://docs.codehaus.org/display/GEOSDOC/Project+Conventions

Under Testing.

Please check them out and make any comments/corrections.

Thanks.

Justin

p.s. If someone was really keen they could write an http unit test which
would churn through all the files in the demo directory.

3

Magne Skjeret wrote:

Justin Deoliveira wrote:

Hi all,

Hi

Excellent work. I have started to do some post vs. get testing for the
GetMap requests.

Oh, it has already been done. :slight_smile:
Well, I will try to add some additional testing.

Magne

I have some questions in that regard:

1. For a test setup is there a map that I can use that I know will be
there? Like states. Or would I have to upload a map in my test suite?

2. I also find it a bit odd that the httptest.properties is commited to
subversion. Each of the developers could have their own setup, e.g.
different port number. Since it is in the code repository, someone are
in danger of committing it making the life of others harder.

My suggestion would be to e.g. add a httptest.properties.example file,
and document that testers must copy that file into httptest.properties
using their own setup parameters.

Magne

I have spent some time playing with HttpUnit
(http://httpunit.sourceforge.net/) and found it to be a very effective
means of testing.

Unit tests only get us so far (we dont have many) as they test things in
isolation. Http unit tests are much like CITE tests in that they are a
good tool for integration testing.

I added the httpunit library to the codebase. There are also some
example tests under the httptest directory. I wanted to seperate them
from the normal unit tests since they require a running instance of
geoserver.

I also wrote some docs on how to execute the tests.

http://docs.codehaus.org/display/GEOSDOC/Project+Conventions

Under Testing.

Please check them out and make any comments/corrections.

Thanks.

Justin

p.s. If someone was really keen they could write an http unit test which
would churn through all the files in the demo directory.

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

4

Hi Magne,

Writing some http tests would be great and is greatly encouraged. I myself am going to try and get into the habit of writing a test case anytime a bug shows up. This will be an effective way to make sure that it never crops up again.

Magne Skjeret wrote:

Justin Deoliveira wrote:

Hi all,

Hi

Excellent work. I have started to do some post vs. get testing for the
GetMap requests.

I have some questions in that regard:

1. For a test setup is there a map that I can use that I know will be
there? Like states. Or would I have to upload a map in my test suite?

In the docs I wrote, I recommend that everyone write tests against the UserBasic cite configuration. I thought this is easiest way to ensure that everyone can run the http unit tests. However I am sure that this configuration might not have the necessary data for a particular test.

I am open to ways to go about this. We could create a new configuration that is soley for testing, but I think that is what UserBasic already is. The easiest thing is probably just to add data to it when needed fo r a test.

2. I also find it a bit odd that the httptest.properties is commited to
subversion. Each of the developers could have their own setup, e.g.
different port number. Since it is in the code repository, someone are
in danger of committing it making the life of others harder.

The default paramters assume the following defaults:

- http protocol
- 8080 port
- localhost
- geoserver context

Which i think is what alot of devlopers do. My intention is to have the httptest.properties be handled like build.properties which has custom paramters in it. In other words, noone should ever commit their httptest.properties file.

However I am open to other ways to do this. Anyone else have any ideas?

My suggestion would be to e.g. add a httptest.properties.example file,
and document that testers must copy that file into httptest.properties
using their own setup parameters.

Magne

I have spent some time playing with HttpUnit
(http://httpunit.sourceforge.net/) and found it to be a very effective
means of testing.

Unit tests only get us so far (we dont have many) as they test things in
isolation. Http unit tests are much like CITE tests in that they are a
good tool for integration testing.

I added the httpunit library to the codebase. There are also some
example tests under the httptest directory. I wanted to seperate them
from the normal unit tests since they require a running instance of
geoserver.

I also wrote some docs on how to execute the tests.

http://docs.codehaus.org/display/GEOSDOC/Project+Conventions

Under Testing.

Please check them out and make any comments/corrections.

Thanks.

Justin

p.s. If someone was really keen they could write an http unit test which
would churn through all the files in the demo directory.

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org

5

Justin Deoliveira wrote:

Hi Magne,

Writing some http tests would be great and is greatly encouraged. I
myself am going to try and get into the habit of writing a test case
anytime a bug shows up. This will be an effective way to make sure that
it never crops up again.

Hi

This has always been my preferred approach, but it is hard to keep up.
I do recognise the advantages with automatic testing, but it takes time
to create good tests. In a stressful environment that a developer lives
in, time limes and delivery dates, it is hard to keep up testing.

In a open source project like this however, I see no excuse for not
testing. Surely there should be some pressure on delivery, but not on
the expense of system reliability.

It would also be possible to use ServletUnit
(http://httpunit.sourceforge.net/doc/servletunit-intro.html) to emulate
the application server. I have not read to much about it yet, but if we
had a proper set-up http unit tests could be executed without any server
running. Sounds neat, but maybe to much work? Will certainly look into
it though.

I have written some tests and the result baffles me a bit:
    [junit] Testcase: testImage took 0,578 sec
    [junit] Testcase: testImagePost took 0,156 sec

The two tests is the exact same. The first is GET and the second is
POST. The result is almost the same each time, but post use almost just
1/3 of the time of get.
I do wonder where the time delta between them come from.
Is it the test set-up, overhead in the http protocol, tomcat parsing the
query string to get the parameters?

I would like to implement POST handling directly in each of the
servlets, and make the TestWfsPost servlet redundant, but not unusable.
A good place to start would be to write the tests for this, making sure
I don't break stuff.

I am not sure how the sample request is populated on the page, but it
would be a good feature to implement all of them in a httpunit test, so
that I/we don't break stuff again.

Next to the ant build file cleanup that I volunteered for, it seems like
that I have my work cut out for me. That is of course if you guys agree
with the work I have outlined.

It is certainly great working on a project, driven only by the
excitement of creating a component a lot of people all around the world
can make use of.

Magne

Magne Skjeret wrote:

Justin Deoliveira wrote:

Hi all,

Hi

Excellent work. I have started to do some post vs. get testing for the
GetMap requests.

I have some questions in that regard:

1. For a test setup is there a map that I can use that I know will be
there? Like states. Or would I have to upload a map in my test suite?

In the docs I wrote, I recommend that everyone write tests against the
UserBasic cite configuration. I thought this is easiest way to ensure
that everyone can run the http unit tests. However I am sure that this
configuration might not have the necessary data for a particular test.

I am open to ways to go about this. We could create a new configuration
that is soley for testing, but I think that is what UserBasic already
is. The easiest thing is probably just to add data to it when needed fo
r a test.

2. I also find it a bit odd that the httptest.properties is commited to
subversion. Each of the developers could have their own setup, e.g.
different port number. Since it is in the code repository, someone are
in danger of committing it making the life of others harder.

The default paramters assume the following defaults:

- http protocol
- 8080 port
- localhost
- geoserver context

Which i think is what alot of devlopers do. My intention is to have the
httptest.properties be handled like build.properties which has custom
paramters in it. In other words, noone should ever commit their
httptest.properties file.

However I am open to other ways to do this. Anyone else have any ideas?

My suggestion would be to e.g. add a httptest.properties.example file,
and document that testers must copy that file into httptest.properties
using their own setup parameters.

Magne

I have spent some time playing with HttpUnit
(http://httpunit.sourceforge.net/) and found it to be a very effective
means of testing.

Unit tests only get us so far (we dont have many) as they test things in
isolation. Http unit tests are much like CITE tests in that they are a
good tool for integration testing.

I added the httpunit library to the codebase. There are also some
example tests under the httptest directory. I wanted to seperate them
from the normal unit tests since they require a running instance of
geoserver.

I also wrote some docs on how to execute the tests.

http://docs.codehaus.org/display/GEOSDOC/Project+Conventions

Under Testing.

Please check them out and make any comments/corrections.

Thanks.

Justin

p.s. If someone was really keen they could write an http unit test which
would churn through all the files in the demo directory.

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log
files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

6

Magne Skjeret wrote:

Justin Deoliveira wrote:

Hi Magne,

Writing some http tests would be great and is greatly encouraged. I
myself am going to try and get into the habit of writing a test case
anytime a bug shows up. This will be an effective way to make sure that
it never crops up again.

Hi

This has always been my preferred approach, but it is hard to keep up.
I do recognise the advantages with automatic testing, but it takes time
to create good tests. In a stressful environment that a developer lives
in, time limes and delivery dates, it is hard to keep up testing.

In a open source project like this however, I see no excuse for not
testing. Surely there should be some pressure on delivery, but not on
the expense of system reliability.

Yeah I know, writing good tests is time consuming, and when you have deliverables to make, they are the first thing to go.

It would also be possible to use ServletUnit
(http://httpunit.sourceforge.net/doc/servletunit-intro.html) to emulate
the application server. I have not read to much about it yet, but if we
had a proper set-up http unit tests could be executed without any server
running. Sounds neat, but maybe to much work? Will certainly look into
it though.

This looks neat, this is more actual "unit" testing as it tests Servlet internals. The normal Http Unit tests could be considered more "integration" testing.

I have written some tests and the result baffles me a bit:
    [junit] Testcase: testImage took 0,578 sec
    [junit] Testcase: testImagePost took 0,156 sec

The two tests is the exact same. The first is GET and the second is
POST. The result is almost the same each time, but post use almost just
1/3 of the time of get.
I do wonder where the time delta between them come from.
Is it the test set-up, overhead in the http protocol, tomcat parsing the
query string to get the parameters?

Hmm this is interesting, it could be that the second time around the post is hitting a cached image or something. I wonder what times you get if you run the tests in isolation.

I would like to implement POST handling directly in each of the
servlets, and make the TestWfsPost servlet redundant, but not unusable.
A good place to start would be to write the tests for this, making sure
I don't break stuff.

Could you elaborate a bit more on your plan here. But yes, writing the tests first would be a great way to ensure that nothing breaks.

I am not sure how the sample request is populated on the page, but it
would be a good feature to implement all of them in a httpunit test, so
that I/we don't break stuff again.

Yes, this is a great idea, the only thing that is missing is the expected result. However we could just run the tests and make sure the server doesn't return a ServiceException element.

Next to the ant build file cleanup that I volunteered for, it seems like
that I have my work cut out for me. That is of course if you guys agree
with the work I have outlined.

An ant build file cleanup would be nice. When I was adding the tasks for the http unit testing I found it to be hard to follow. Personally I would like to see the ant build file replaced with a maven build. The ant scripts sets up a lot of paths/locations/etc... and maven already assumes good defaults so it would be a lot less verbose. Just an idea though, I am not sure that many other people share my love of maven.

It is certainly great working on a project, driven only by the
excitement of creating a component a lot of people all around the world
can make use of.

I am glad you are enjoying working on GeoServer. You have made a great addition to the development team. Keep up the good work!!

-Justin

Magne

Magne Skjeret wrote:

Justin Deoliveira wrote:

Hi all,

Hi

Excellent work. I have started to do some post vs. get testing for the
GetMap requests.

I have some questions in that regard:

1. For a test setup is there a map that I can use that I know will be
there? Like states. Or would I have to upload a map in my test suite?

In the docs I wrote, I recommend that everyone write tests against the
UserBasic cite configuration. I thought this is easiest way to ensure
that everyone can run the http unit tests. However I am sure that this
configuration might not have the necessary data for a particular test.

I am open to ways to go about this. We could create a new configuration
that is soley for testing, but I think that is what UserBasic already
is. The easiest thing is probably just to add data to it when needed fo
r a test.

2. I also find it a bit odd that the httptest.properties is commited to
subversion. Each of the developers could have their own setup, e.g.
different port number. Since it is in the code repository, someone are
in danger of committing it making the life of others harder.

The default paramters assume the following defaults:

- http protocol
- 8080 port
- localhost
- geoserver context

Which i think is what alot of devlopers do. My intention is to have the
httptest.properties be handled like build.properties which has custom
paramters in it. In other words, noone should ever commit their
httptest.properties file.

However I am open to other ways to do this. Anyone else have any ideas?

My suggestion would be to e.g. add a httptest.properties.example file,
and document that testers must copy that file into httptest.properties
using their own setup parameters.

Magne

I have spent some time playing with HttpUnit
(http://httpunit.sourceforge.net/) and found it to be a very effective
means of testing.

Unit tests only get us so far (we dont have many) as they test things in
isolation. Http unit tests are much like CITE tests in that they are a
good tool for integration testing.

I added the httpunit library to the codebase. There are also some
example tests under the httptest directory. I wanted to seperate them
from the normal unit tests since they require a running instance of
geoserver.

I also wrote some docs on how to execute the tests.

http://docs.codehaus.org/display/GEOSDOC/Project+Conventions

Under Testing.

Please check them out and make any comments/corrections.

Thanks.

Justin

p.s. If someone was really keen they could write an http unit test which
would churn through all the files in the demo directory.

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log
files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org

7

Magne Skjeret wrote:

It would also be possible to use ServletUnit
(http://httpunit.sourceforge.net/doc/servletunit-intro.html) to emulate
the application server. I have not read to much about it yet, but if we
had a proper set-up http unit tests could be executed without any server
running. Sounds neat, but maybe to much work? Will certainly look into
it though.

Jetty (what GeoServer uses right now), was also developed w/ this in mind....

I have written some tests and the result baffles me a bit:
   [junit] Testcase: testImage took 0,578 sec
   [junit] Testcase: testImagePost took 0,156 sec

The two tests is the exact same. The first is GET and the second is
POST. The result is almost the same each time, but post use almost just
1/3 of the time of get.
I do wonder where the time delta between them come from.
Is it the test set-up, overhead in the http protocol, tomcat parsing the
query string to get the parameters?

Probably the JIT kicking in, always run your code 3 times before doing any timing. I am sure testImage calls a lot
of the same code that testImagePost is using, testImagePost is just benifiting from a bit of compiled code...

You can also profile the two, just profile an application like normal - the application just happens to be Jetty. Eralier when we ran Resin there was a lot of support with an Eclipse plugin...
Jody

8

Justin Deoliveira wrote:

Hi
(Not sure if it is good practice, but I have copied a part of the thread
to the top, it was just to crowded down there).

I would like to implement POST handling directly in each of the
servlets, and make the TestWfsPost servlet redundant, but not
unusable.
A good place to start would be to write the tests for this, making
sure
I don't break stuff.

Could you elaborate a bit more on your plan here. But yes, writing the
tests first would be a great way to ensure that nothing breaks.

It is not much really. I just want to be able to post from web-pages,
like get, without taking the roundtrip with the TestWfsPost servlet.
I am sorry to have made this servlet my enemy number 1, but I fail to
see the reason why it is there.
There might of course be some historical reasons for it, but a
geoserver-newbie like me would not know.
I suspect it might have something to do with how the sample request page
is populated, but I am not sure about that.

I am not planning to remove it or anything, just add the possibility to
do it directly from a browser, just as has been done for GetMap.
It is something I would like to do, mostly to get familiar with the code.
If this is something you don't want in yet, or at all, I don't mind not
doing it.

Magne

Old thread:

Magne Skjeret wrote:

Justin Deoliveira wrote:

Hi Magne,

Writing some http tests would be great and is greatly encouraged. I
myself am going to try and get into the habit of writing a test case
anytime a bug shows up. This will be an effective way to make sure that
it never crops up again.

Hi

This has always been my preferred approach, but it is hard to keep up.
I do recognise the advantages with automatic testing, but it takes time
to create good tests. In a stressful environment that a developer lives
in, time limes and delivery dates, it is hard to keep up testing.

In a open source project like this however, I see no excuse for not
testing. Surely there should be some pressure on delivery, but not on
the expense of system reliability.

Yeah I know, writing good tests is time consuming, and when you have
deliverables to make, they are the first thing to go.

It would also be possible to use ServletUnit
(http://httpunit.sourceforge.net/doc/servletunit-intro.html) to emulate
the application server. I have not read to much about it yet, but if we
had a proper set-up http unit tests could be executed without any server
running. Sounds neat, but maybe to much work? Will certainly look into
it though.

This looks neat, this is more actual "unit" testing as it tests Servlet
internals. The normal Http Unit tests could be considered more
"integration" testing.

I have written some tests and the result baffles me a bit:
    [junit] Testcase: testImage took 0,578 sec
    [junit] Testcase: testImagePost took 0,156 sec

The two tests is the exact same. The first is GET and the second is
POST. The result is almost the same each time, but post use almost just
1/3 of the time of get.
I do wonder where the time delta between them come from.
Is it the test set-up, overhead in the http protocol, tomcat parsing the
query string to get the parameters?

Hmm this is interesting, it could be that the second time around the
post is hitting a cached image or something. I wonder what times you get
if you run the tests in isolation.

I would like to implement POST handling directly in each of the
servlets, and make the TestWfsPost servlet redundant, but not unusable.
A good place to start would be to write the tests for this, making sure
I don't break stuff.

Could you elaborate a bit more on your plan here. But yes, writing the
tests first would be a great way to ensure that nothing breaks.

I am not sure how the sample request is populated on the page, but it
would be a good feature to implement all of them in a httpunit test, so
that I/we don't break stuff again.

Yes, this is a great idea, the only thing that is missing is the
expected result. However we could just run the tests and make sure the
server doesn't return a ServiceException element.

Next to the ant build file cleanup that I volunteered for, it seems like
that I have my work cut out for me. That is of course if you guys agree
with the work I have outlined.

An ant build file cleanup would be nice. When I was adding the tasks for
the http unit testing I found it to be hard to follow. Personally I
would like to see the ant build file replaced with a maven build. The
ant scripts sets up a lot of paths/locations/etc... and maven already
assumes good defaults so it would be a lot less verbose. Just an idea
though, I am not sure that many other people share my love of maven.

It is certainly great working on a project, driven only by the
excitement of creating a component a lot of people all around the world
can make use of.

I am glad you are enjoying working on GeoServer. You have made a great
addition to the development team. Keep up the good work!!

-Justin

Magne

Magne Skjeret wrote:

Justin Deoliveira wrote:

Hi all,

Hi

Excellent work. I have started to do some post vs. get testing for the
GetMap requests.

I have some questions in that regard:

1. For a test setup is there a map that I can use that I know will be
there? Like states. Or would I have to upload a map in my test suite?

In the docs I wrote, I recommend that everyone write tests against the
UserBasic cite configuration. I thought this is easiest way to ensure
that everyone can run the http unit tests. However I am sure that this
configuration might not have the necessary data for a particular test.

I am open to ways to go about this. We could create a new configuration
that is soley for testing, but I think that is what UserBasic already
is. The easiest thing is probably just to add data to it when needed fo
r a test.

2. I also find it a bit odd that the httptest.properties is commited to
subversion. Each of the developers could have their own setup, e.g.
different port number. Since it is in the code repository, someone are
in danger of committing it making the life of others harder.

The default paramters assume the following defaults:

- http protocol
- 8080 port
- localhost
- geoserver context

Which i think is what alot of devlopers do. My intention is to have the
httptest.properties be handled like build.properties which has custom
paramters in it. In other words, noone should ever commit their
httptest.properties file.

However I am open to other ways to do this. Anyone else have any ideas?

My suggestion would be to e.g. add a httptest.properties.example file,
and document that testers must copy that file into httptest.properties
using their own setup parameters.

Magne

I have spent some time playing with HttpUnit
(http://httpunit.sourceforge.net/) and found it to be a very effective
means of testing.

Unit tests only get us so far (we dont have many) as they test
things in
isolation. Http unit tests are much like CITE tests in that they are a
good tool for integration testing.

I added the httpunit library to the codebase. There are also some
example tests under the httptest directory. I wanted to seperate them
from the normal unit tests since they require a running instance of
geoserver.

I also wrote some docs on how to execute the tests.

http://docs.codehaus.org/display/GEOSDOC/Project+Conventions

Under Testing.

Please check them out and make any comments/corrections.

Thanks.

Justin

p.s. If someone was really keen they could write an http unit test
which
would churn through all the files in the demo directory.

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log
files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel