ctest_test: Add option to specify configuration in which to run tests
In a dashboard script, there is no way to select the build configuration for the test step, which is required for multi config generators.
The following dashboard script reproduces the issue:
#!/usr/bin/env -S ctest -VV -S
cmake_minimum_required(VERSION 3.20)
set(CTEST_SOURCE_DIRECTORY ".")
set(CTEST_BINARY_DIRECTORY "./build")
set(CTEST_CMAKE_GENERATOR "Ninja Multi-Config") # or VS or Xcode
ctest_start("Experimental")
ctest_configure()
ctest_build()
ctest_test()
When using a multi-config generator, like Ninja Multi-Config, Visual Studio, or Xcode, the test step fails with error messages like:
Test not available without configuration. (Missing "-C <config>"?)
The ctest_build
command accepts a CONFIGURATION
argument, so we can build multiple configurations (When no CONFIGURATION
is passed to ctest_build
, it falls back to CTEST_BUILD_CONFIGURATION
, CTEST_CONFIGURATION_TYPE
, the -C
command line argument, or the hardcoded value "Release"):
ctest_build(CONFIGURATION Debug)
ctest_build(CONFIGURATION Release)
The ctest_test
command does not accept a CONFIGURATION
argument and can only be controlled with the -C
command line argument (which only accepts a single value, so it defeats the purpose of a multi-config generator):
ctest_test(CONFIGURATION Debug) # Does not work
ctest_test(CONFIGURATION Release) # Does not work
Ideally, the test step should accept a CONFIGURATION
option. And when the option is not given, it should mirror the fallback logic of the build step.
All of the above also applies to the memcheck step.