ctest: "Not Run" tests should not be listed among "Failed" tests
Isolated issue from #17980
If one marks a test as skipped via SKIP_RETURN_CODE
, and tries to also skip its dependent tests via the DEPENDS
and REQUIRED_FILES
properties (here named: diags
), the dependent tests are categorized as failed tests and thus falsely fail ctest
(non-zero return code).
Internal ctest changing into directory: ...
Test project ...
Start 182: my_test.run
1/3 Test #182: my_test.run ........***Skipped 1.81 sec
Start 183: my_test.analysis
Unable to find required file: diags
2/3 Test #183: my_test.analysis ...***Not Run 0.00 sec
Start 184: my_test.plot
Unable to find required file: diags
3/3 Test #184: my_test.plot .......***Not Run 0.00 sec
33% tests passed, 2 tests failed out of 3
Label Time Summary:
slow = 1.81 sec*proc (3 tests)
Total Test time (real) = 1.82 sec
The following tests did not run:
182 - my_test.run (Skipped)
The following tests FAILED:
183 - my_test.analysis (Not Run)
184 - my_test.plot (Not Run)
Errors while running CTest
Output from these tests are in: .../Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
I think the latter should be listed under did not run
, too.
Similar to not having the latter two tests, and only the one skipped test, the return code of ctest should be 0.
Edited by Axel Huebl