It was not required by GSoC but I wanted to do some final report to summarize what was done and what could be done in the future. So, here is a shorter version of the report, the full is on Trac.
Ways to run tests:
- each file separately in the current location and mapset you are in
- good for testing the code you are just writing
- good for writing the test
- running all tests in subdirectories
- generates a report
- switches to specified location and mapset
- generates detailed report
- using higher level script to run tests from outside GRASS session
How to write tests:
- using GRASS Python package
gunittest - works in the same way as
unittest - adds GRASS-specific functionality
- allows generation of HTML reports
- enables fine control over execution of GRASS modules
- this is the preferred way, other ways such Shell scripts are discouraged
When writing tests, gunittest package allows in a convenient way to:
- compare map statistics or info to a given reference
- compare raster and 3D raster maps with each other
- compare vector maps using different ways
- compare numbers and texts
Things which should be solved in the short term:
- tests should specify the location they can run in
- encapsulate removal of maps and other files
- test tests on more platforms
Long-term plans:
-
integrate tools such as
valgrind -
provide coverage
-
introduce static test of source code
-
time execution of modules
It is probably good to mention that documentation with examples is in Python API Sphinx documentation. You can get it by make sphinxdoc.
And there is probably no need to mention that although there is a lot of tests which can serve you as examples there is not enough tests to cover all GRASS functionality, so please contribute.
Thanks to the test, we have already discovered several bugs soon after they were introduced (the day of after the commit). This is very encouraging to me and I hope also to others.
If you have some feedback or problems with writing tests, please share.
Vaclav
http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS#Finalreport