ctest --parallel doesn't maximally parallelize my XCTests.
When I When I do swift test --parallel
on a Swift Package Manager project and look at it with ps waux | grep xctest
, I see one xctest
invocation per core and individual test functions running in parallel. Invocations look like this:
/path/to/xctest -XCTest EndToEndTests.EndToEndTests/test_compileAndRun_Fibonacci /my-project/.build/arm64-apple-macosx/debug/HyloPackageTests.xctest
where test_compileAndRun_Fibonacci
is the name of a test function. It tests my project's debug build in 176s on my machine.
But when I build the same project under cmake and use ctest --parallel
I see invocations like this:
/path/to/xctest /my-project/.ninja-build/Tests/EndToEndTests.xctest
Invoking the entire xctest in a single process. There are a few long-running xctest targets in my project, but not enough to keep my cores occupied. So the entire test run takes 612s. We should find a way to overcome this limitation.
At least on macOS, interrogating the list of test functions in a compiled XCTest is possible. The problem may be harder on other platforms, but on those platforms the list needs to be maintained by hand anyway (or scraped from the sources automatically as part of the build process). So we could arrange for that list to become available to ctest
.