I feel like I could write a book on the cite engine 
but I will keep it short.
* Automation
As you noticed, quite painful. There is no good way to automate a test run. I did a while back hack in a way to run the engine "headless", that is supply the input parameters via a xml file rather than pop up a gui. I think i have this patch somewhere on my system....
* Performance
Horrid. The engine by definition is slow being purely XSLT based. If you run the engine from the command line you pay a 1-2 minute hit on setup time. If you run the engine via webapp you only pay the cost once. However the only way we have found to do this without having it run out of memory is to ensure that you include only those tests for the suite you need, and not all of them. However the folks a cite.opengeospatial.org seem to have a way around this. Not sure what it is.
* Debugging
Debugging the tests is again quite painful, again due mostly to the fact that the tests are XSLT based. So you are stuck with the brute force approach of simply outputting values. And since recompiling the tests takes literally minutes this is cumbersome to say the least.
* Extensibility
It looks easy enough... but I have never actually written my own test case.
The only other plus side to the engine is really the number of test cases that have been written for it. There are lots of WFS test cases and they serve as quite a nice regression testing tool.
So basically if you have a XSLT guru in your shop (wink wink) the CITE engine is a decent option. However I would love to see an effort to port the engine to a nicer framework, something like junit or testng. But that would take a lot of work unfortunately.
-Justin
Ben Caradoc-Davies wrote:
I have been asked my opinions on the TEAM engine used for CITE tests. In the future we can expect more CITE tests, so it might be useful to ensure development is informed by our requirements.
The TEAM Engine forced me to use an interactive GUI, which I did not like. I would prefer something easier to automate and integrate with automated testing systems. This would allow us to set up, for example, a CITE Hudson. Is this easy to do with the TEAM Engine?
Does anyone else have any opinions in the utility of the TEAM Engine currently used for the CITE tests?
Kind regards,
--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.