My question is, what is the appropriate solution here:
split up the test, so that tests savely finish within 300 seconds threshold? The drawback is that the creation of testdata in those tests often takes ~ 20 to 50 seconds, which is an overhead in each testfile o
increase the time-out limit a bit, so the test does not fail…
Any temporal test can fail with timeout. Not only this one. I think this one is most commonly failing because it has the most tests, not because it is more wrong then the others.
My question is, what is the appropriate solution here:
split up the test, so that tests savely finish within 300 seconds threshold? The drawback is that the creation of testdata in those tests often takes ~ 20 to 50 seconds, which is an overhead in each testfile o
I think the split would just increase the time for good cases and while the number of processes which may hang will remain the same.
increase the time-out limit a bit, so the test does not fail…
I think the tests hang, not simply take a little longer. Before the time limit, they would take (the full) 6 hours. So, increasing the test time limit would make it fail later, but that’s all.
The actual problem is unknown, but I don’t know if anyone tried like 100-500 tests in a row locally to see what happens. The test results are available for download even when the tests fail (to help track down this problem).