ctest v3.27.x: ignores --timeout option if option is greater than default
ctest exposes a --timeout option. Since using ctest v3.27.1, we have observed that:
[ok] if the option is not supplied then ctest uses a default value of 1500 [ok] if 'seconds <= 1500'... then ctest respects the user-supplied timeout [regression] if 'seconds > 1500' then ctest "clamps" the timeout at 1500
Steps to reproduce actual behavior in ctest v3.27.1
Using
cmake_minimum_required( VERSION 3.13 )
project( ctest-timeout LANGUAGES CXX )
include( CTest )
add_test(NAME test-timeout COMMAND echo "no-op")
$ cd ~/playground/ctest-timeout/build
$ /bin/cmake-3.27.1/ctest --version
ctest version 3.27.1
CMake suite maintained and supported by Kitware (kitware.com/cmake).
$ /bin/cmake-3.27.1/ctest -V --timeout 1501
UpdateCTestConfiguration from :~/playground/ctest-timeout/build/DartConfiguration.tcl
Parse Config file:~/playground/ctest-timeout/build/DartConfiguration.tcl
UpdateCTestConfiguration from :~/playground/ctest-timeout/build/DartConfiguration.tcl
Parse Config file:~/playground/ctest-timeout/build/DartConfiguration.tcl
Test project ~/playground/ctest-timeout/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
Start 1: test-timeout
1: Test command: /bin/echo "no-op"
1: Working Directory: ~/playground/ctest-timeout/build
1: Test timeout computed to be: 1500
1: no-op
1/1 Test #1: test-timeout ..................... Passed 0.02 sec
100% tests passed, 0 tests failed out of 1
Total Test time (real) = 0.03 sec
$ /bin/ctest -V --timeout 1499
UpdateCTestConfiguration from :~/playground/ctest-timeout/build/DartConfiguration.tcl
Parse Config file:~/playground/ctest-timeout/build/DartConfiguration.tcl
UpdateCTestConfiguration from :~/playground/ctest-timeout/build/DartConfiguration.tcl
Parse Config file:~/playground/ctest-timeout/build/DartConfiguration.tcl
Test project ~/playground/ctest-timeout/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
Start 1: test-timeout
1: Test command: /bin/echo "no-op"
1: Working Directory: ~/playground/ctest-timeout/build
1: Test timeout computed to be: 1499
1: no-op
1/1 Test #1: test-timeout ..................... Passed 0.02 sec
100% tests passed, 0 tests failed out of 1
Total Test time (real) = 0.03 sec
Steps to reproduce expected behavior in ctest 3.26.2
$ cd ~/playground/ctest-timeout/build
$ /bin/cmake-3.26.2/ctest --version
ctest version 3.26.2
CMake suite maintained and supported by Kitware (kitware.com/cmake).
$ /bin/cmake-3.26.2/ctest -V --timeout 1501
UpdateCTestConfiguration from :~/playground/ctest-timeout/build/DartConfiguration.tcl
Parse Config file:~/playground/ctest-timeout/build/DartConfiguration.tcl
UpdateCTestConfiguration from :~/playground/ctest-timeout/build/DartConfiguration.tcl
Parse Config file:~/playground/ctest-timeout/build/DartConfiguration.tcl
Test project ~/playground/ctest-timeout/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
Start 1: test-timeout
1: Test command: /bin/echo "no-op"
1: Working Directory: ~/playground/ctest-timeout/build
1: Test timeout computed to be: 1501
1: no-op
1/1 Test #1: test-timeout ..................... Passed 0.03 sec
100% tests passed, 0 tests failed out of 1
Total Test time (real) = 0.04 sec
Unfortunately, cmake 3.27.6 did not fix the issue.