FindMatlab matlab_add_unit_test reports exception as Passed
If the option NO_UNITTEST_FRAMEWORK is used, then MatlabTestsRedirect.cmake
correctly puts the call to the test function in a try/catch block and returns with error status in the case of an exception. However, if the unit testing framework is used, it just calls runtests('$unittest_file')
. If this throws an exception it is ignored. We subsequently return with status 0 and ctest says the test is Passed.
It is easy to produce such an exception by passing a file which does not exist to UNITTEST_FILE. Or in my case I neglected to prefix the file path with ${CMAKE_CURRENT_SOURCE_DIR}
. Furthermore, although runtests catches exceptions from the body of the test, I did encounter some less trivial cases where runtests produced an uncaught exception.
Below is an example output and CMakeLists.txt for reproducibility.
$ ctest3 --verbose .
[...]
1: Error using testsuite (line 99)
1: "doesNotExist" is not recognized as a test entity.
1:
1: Error in runtests (line 69)
1: suites = testsuite(parser.Results.tests,pvcell{:});
1:
1:
1/1 Test #1: mytest ........................... Passed 5.08 sec
100% tests passed, 0 tests failed out of 1
CMakeLists.txt:
cmake_minimum_required(VERSION 3.6)
find_package(Matlab REQUIRED MAIN_PROGRAM)
enable_testing()
matlab_add_unit_test(NAME mytest UNITTEST_FILE doesNotExist.m)