[GRASS-dev] Write a test to proof that the commit works

Hi all,

as you know, I’m working on a GSoC project implementing a testing framework for GRASS [1]. I’m at the beginning but I want to tell you that you can start to write test right now as I recently did for g.list [2]. If you use this as a template, you will ensure that it will be compatible with the framework I’m working on.

Some projects require to have a test (test case or whatever) for each newly added feature so that it is clear that the feature works and it will be clear when it breaks (see r606618 for example). In other words, the change/commit/patch is not accepted if it is not accompanied by a test (if a test does not exists already). In the combination with tests running after each commit, this is much more powerfull then reading commits or svn blame because it marks the commits which should be reverted if tests fail.

I don’t think that we need to set up rules like this but I would like to encourage you to do it, to write the tests for your changes, especially for the commits which are changing core functionality and the test is not hard to write (the things which need to be tested are text or few numbers, not maps, for example [4]).

For example to test the functionality of the parser, the simplest solution I see is to test some module which is using the parser’s feature by running module with run_command and testing the return code using assertEquals. There is few other possibilities but anything will do.

Please note that I’m not be able to write the actual tests for the whole GRASS, not even for significant part of it. The topic of GSoC is a framework which should provide good tools, environment settings, reports and so on. The good tools may simplify test writing and will make it more precise but nothing will save us from writing the tests for our work, our changes and the functionality we are relying on.

Feel free to ask me questions here or privately about writing tests. I will try to answer as soon as possible and reflect you comments in the documentation and the code itself.

Sorry for such a long email but I would like to motivate you to contribute a really long list of long tests.

Vaclav

[1] http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS
[2] http://trac.osgeo.org/grass/browser/grass/trunk/general/g.list/test_g_list.py?rev=60619 (currently broken I guess :slight_smile:
[3] http://trac.osgeo.org/grass/changeset/60618
[4] http://trac.osgeo.org/grass/changeset/60713

PS: I have some parts of the framework already in the sandbox [5] but it is unstable and for writing tests you can do the same with standard Python unittest library [6].

[5] http://trac.osgeo.org/grass/browser/sandbox/wenzeslaus/gunittest
[6] https://docs.python.org/2/library/unittest.html