CTest: Success reported after FATAL_ERROR from CMake
This ticket is written after discussion in: !4078 (merged).
A change was made there to allow the gtest_discover_tests function to perform its test discovery at test runtime instead of at build time. In the case that the test discovery fails (for example because the test binary crashes) the gtest_discover_tests function will exit with a FATAL_ERROR. Unfortunately this is not detected by CTest which reports
No tests were found!!! and then exits with a zero exit-code (success). This is problematic in build pipelines as faulty tests may not be detected (i.e. the build could be marked as success).
Known workaround is to invoke ctest with the
--no-tests=error flag but that is just checking for a symptom of the above behaviour and cannot be considered a real fix.
Suggestions from the discussion linked to above are:
- Improve CTest error detection to pick up on the FATAL_ERROR coming from the gtest_discover_tests function
- Make the gtest_discover_tests function add a dummy failing test in cases where the test binary contains errors