[GRASS-dev] Re: GRASS Test suite

GRASS GRASS developer
1

Hi Anne,
i will put the GRASS test suite implementation discussion on the dev
list if you don't mind? :slight_smile:

A segfault should not happen. All tests run on my systems very well
(latest grass7, openSuse 11.2).
Do you use the latest grass7 trunk? Can you please use gdb to hunt
down the error, i will fix it.

I use GRASS 7 r 46894, compiled this morning, on Fedora 14 64bit.

`r3.colors -e map=volume_double_null rules=example1`

this command segfaults at raster/r.colors/stats.c, line 155, with
invalid pointer:

G_free(dcell);

Markus looked at the problem and suggested this patch, line 150:

if (type == RASTER_TYPE) {
Rast_close(fd);
G_free(dcell);
}

(sorry for patch format but it's such a tiny modification)

Many thanks for the patch, i have committed your solution to svn. Good
to have some tests. :slight_smile:

I see that in the wiki the part about Python has been removed, which
parts of the system we drafted in Prague are still valid?

Not much. :slight_smile:

I'm fine with setting up the Python part of the framework. I fear that
as I'm not familiar with makefiles, it's better to ask Glynn and the dev
list for guidance too.

I have updated the wiki with more implementation details for the
Python test framework:

http://grass.osgeo.org/wiki/Test_Suite#Implementation

This is just a very draft concept but it hopefully shows the
implementation approach i have in mind. IMHO the implementation should
not be very complex, except the ability of the framework to act as a
simple shell.

Regarding the Make system: unfortunately i am not familiar with
Makefile handling too, so i am no help on this topic.

Should we move the wiki page to the new trac wiki or can we leave it
in the old? I would suggest to move ... .

Best regards
Soeren

2

Hi,

can we move the test files into $MODULE/testsuite/ or similar?

I'm just looking at grass7/raster/r.colors/ and the test files
really dominate the module dir.

thanks,
Hamish

3

Le dimanche 03 juillet 2011 à 15:53 -0700, Hamish a écrit :

Hi,

can we move the test files into $MODULE/testsuite/ or similar?

I'm just looking at grass7/raster/r.colors/ and the test files
really dominate the module dir.

+1 from me on having a testsuite/ or tests/ subdir for each module.

I'm starting to implement the test framework, it will be in
trunk/testsuite. Hints on how to modify the Makefile are welcome!

best,
Anne

4

Hi,

2011/7/4 Anne Ghisla <a.ghisla@gmail.com>:

I'm just looking at grass7/raster/r.colors/ and the test files
really dominate the module dir.

+1 from me on having a testsuite/ or tests/ subdir for each module.

also +1 from me.

Martin

--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa

5

Hi all,

2011/7/4 Martin Landa <landa.martin@gmail.com>:

Hi,

2011/7/4 Anne Ghisla <a.ghisla@gmail.com>:

I'm just looking at grass7/raster/r.colors/ and the test files
really dominate the module dir.

+1 from me on having a testsuite/ or tests/ subdir for each module.

also +1 from me.

The directory layout has been discussed here:
http://osgeo-org.1803224.n2.nabble.com/Test-suite-for-GRASS-proposal-discussion-welcome-td6391218.html

I would suggest to either use reference_data/ subdir for each module,
so that the test scripts may stay in the module directory, not in a
subdirectory, or a testsuite/ subdir including all data?

I personally prefer a reference data subdir, to have a clear
separation of test case scripts and reference data. Otherwise the
reference data will dominate again and finding the test scripts is not
that obvious.

Best regards
Sören

Martin

--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

6

Hi Anne,

2011/7/4 Anne Ghisla <a.ghisla@gmail.com>:

Le dimanche 03 juillet 2011 à 15:53 -0700, Hamish a écrit :

Hi,

can we move the test files into $MODULE/testsuite/ or similar?

I'm just looking at grass7/raster/r.colors/ and the test files
really dominate the module dir.

+1 from me on having a testsuite/ or tests/ subdir for each module.

I'm starting to implement the test framework, it will be in
trunk/testsuite. Hints on how to modify the Makefile are welcome!

Thank you for starting the implementation. If you have any tiny
question, don't bother to ask. I hope the whole test suite picture
comes much clearer when some basic framework is available to play
around a with.

Important: in case the test script approach which i have used for the
test cases in the g3d modules is not sufficient, i will modify them to
fit the new guidelines and insights.

Best regards
Soeren

best,
Anne

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

7

Le lundi 04 juillet 2011 à 10:52 +0200, Soeren Gebbert a écrit :

Hi all,

2011/7/4 Martin Landa <landa.martin@gmail.com>:
> Hi,
>
> 2011/7/4 Anne Ghisla <a.ghisla@gmail.com>:
>>> I'm just looking at grass7/raster/r.colors/ and the test files
>>> really dominate the module dir.
>>
>> +1 from me on having a testsuite/ or tests/ subdir for each module.
>
> also +1 from me.

The directory layout has been discussed here:
http://osgeo-org.1803224.n2.nabble.com/Test-suite-for-GRASS-proposal-discussion-welcome-td6391218.html

I would suggest to either use reference_data/ subdir for each module,
so that the test scripts may stay in the module directory, not in a
subdirectory, or a testsuite/ subdir including all data?

I personally prefer a reference data subdir, to have a clear
separation of test case scripts and reference data. Otherwise the
reference data will dominate again and finding the test scripts is not
that obvious.

In the case of r.colors, there are a lot of .ref files and one .sh
script, that is hidden among them when the folders contents are ordered
alphabetically. I haven't checked if it will be the case of other
modules.

Personally I prefer to keep all test data and scripts together in the
tests/ subfolder.
But if the majority find it uncomfortable as the script file is less
accessible, let's keep it in module folder and put data in the subfolder
(after correcting the paths in script file).
Opinions?

regards
Anne

Best regards
Sören

>
> Martin
>
> --
> Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa
> _______________________________________________
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev
>

8

Hello Anne,

2011/7/4 Anne Ghisla <a.ghisla@gmail.com>:

Le lundi 04 juillet 2011 à 10:52 +0200, Soeren Gebbert a écrit :

Hi all,

2011/7/4 Martin Landa <landa.martin@gmail.com>:
> Hi,
>
> 2011/7/4 Anne Ghisla <a.ghisla@gmail.com>:
>>> I'm just looking at grass7/raster/r.colors/ and the test files
>>> really dominate the module dir.
>>
>> +1 from me on having a testsuite/ or tests/ subdir for each module.
>
> also +1 from me.

The directory layout has been discussed here:
http://osgeo-org.1803224.n2.nabble.com/Test-suite-for-GRASS-proposal-discussion-welcome-td6391218.html

I would suggest to either use reference_data/ subdir for each module,
so that the test scripts may stay in the module directory, not in a
subdirectory, or a testsuite/ subdir including all data?

I personally prefer a reference data subdir, to have a clear
separation of test case scripts and reference data. Otherwise the
reference data will dominate again and finding the test scripts is not
that obvious.

In the case of r.colors, there are a lot of .ref files and one .sh
script, that is hidden among them when the folders contents are ordered
alphabetically. I haven't checked if it will be the case of other
modules.

This is the case for most modules with tests applied:
http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.out.ascii
http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.cross.rast
http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.retile
http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.stats
http://trac.osgeo.org/grass/browser/grass/trunk/raster3d/r3.to.rast
http://trac.osgeo.org/grass/browser/grass/trunk/raster/r.to.rast3elev

Personally I prefer to keep all test data and scripts together in the
tests/ subfolder.
But if the majority find it uncomfortable as the script file is less
accessible, let's keep it in module folder and put data in the subfolder
(after correcting the paths in script file).
Opinions?

The paths used in the test scripts should not be affected by the
location of the reference files. Only the framework and the test
designer need to know that. The user/dev who implements the tests must
assure that the reference files are located in the reference
directory. The framework than take care that while testing generated
files are compared with the reference files located in the reference
directory.

Hmmmm, ... i think the test scripts should always write any ascii file
output in the current working directory, so the framework can execute
the tests in a temporary directory which can be removed for clean up
after reference data validation. I will add this to the wiki.

Lets keep for practical reason the test scrip in the module directory
and add a "test_reference_data" or "test_refs" directory to the module
directory keeping the reference data.

Best regards
Sören