ctest: Itemized/Individual test log files
Feature Request - Individual itemized log files
Problem:
CTest outputs one unified log file when running "-O output.log". It's hard to debug individual tests and see the individual problem in the each test.
Proposal
An option to:
- Generate a log file per CTest test into a log directory.
- Specify whether to generate log files for all tests or only failing tests.
- Add test log path to test report that outputs to terminal
Test Report output
Output report would report the failing tests like so:
The following tests FAILED:
1 - bf16_test0 (Failed)
Log: (<full_path>/logs.0/bf16_test0.log)
2 - bf16_test1(Failed)
Log: (<full_path>/logs.1/bf16_test1.log)
*Test Logging output
Each CTest run (CTest_test() or ctest CLI command) would create a folder in "Testing" (Testing/Logs.n) Each test would output a log, testname.log A symlink called "logs_latest" points to the latest log after each test run.
Testing/logs_latest
Testing/logs.0/test_name0.log
Testing/logs.0/test_name1.log
Testing/logs.0/test_name2.log
Testing/logs.1/test_name0.log
Testing/logs.1/test_name1.log
Testing/logs.1/test_name2.log
Configuration
Enabling
** Command Line **
Enable itemized logs for all tests
ctest --itemized-logs=all
Enable itemized logs for failing tests
ctest --itemized-logs=failures
** CMAKE VARIABLE ** (For ctest_test() command)
Enable itemized logs for all tests
set(CTEST_ITEMIZED_LOGS, "ALL")
Enable itemized logs for failing tests
set(CTEST_ITEMIZED_LOGS, "FAILURES")
Edited by Evie Schwarz