[GRASS-dev] Should we become stricter with failures in addon CI?

GRASS GRASS developer
1

Hei,

based on recent experience I wonder if we should`nt become more picky with regards to failing tests for addons.
Test success rate is at ~70 % and if I add a new failing test, that is not considered a test failure, so I have to go in and check if the added test actually succeeded.
I understand that we probably should not be too strict with addon code, but with the current state (with tests silently failing) we get less out of tests than we could do…

Cheers
Stefan

2

Hi Stefan,

On Wed, 5 Oct 2022 at 07:18, Stefan Blumentrath <Stefan.Blumentrath@gmx.de> wrote:

Test success rate is at ~70 % and if I add a new failing test, that is not considered a test failure, so I have to go in and check if the added test actually succeeded.

The tooling improved since the rule was set up, so I would say, yes, it should mirror the setup in the core where not-yet-fixed tests are excluded in config.

Best,

Vaclav