ExternalProject with Ninja generator doesn't provide meaningful build status during build
I am migrating away from an internal build system to cmake, and one of the major requirements that my management has given me is that a single build must be able to build all supported configurations. This is what our current tool uses, and other parts of our CI system would need major changes to support a different approach.
For example purposes, these are the 8 required targets
- windows/{x86, AMD64}/{debug,release}
- linux/{AMD64, AARCH64}/{debug,release}
I have toolchain files for each of these, and can build them all individually.
Initially, I tried to use CMake with the Ninja generator, and generate all 8 configurations with a script, and then call a hand-written build.ninja file that was essentially the following:
subninja cmake-build-windows-x86-debug/build.ninja
subninja cmake-build-windows-x86-release/build.ninja
...
subninja cmake-build-linux-aarch64-release/build.ninja
But that doesn't work due to problems with the current working directory when using the CMake generated build.ninja files this way.
So my next approach was to use the ExternalProject feature. This actually works, and unless a clearly better method is found (happy to hear suggestions) it's what I'll go with. I'm currently using this top-level CMakeLists.txt
cmake_minimum_required(VERSION 3.19.0)
project(PROJ NONE)
if(${PROJECT_NAME}_BUILD_ONE_CONFIG)
enable_testing()
add_subdirectory(3rdparty)
add_subdirectory(main)
add_subdirectory(plugins)
else()
message("Configuring external project")
include(ExternalProject)
macro(setup TOOLCHAIN BUILD_TYPE)
ExternalProject_Add(${TOOLCHAIN}_${BUILD_TYPE}
SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}"
CMAKE_ARGS
-DCMAKE_TOOLCHAIN_FILE=toolchains/${TOOLCHAIN}.cmake
-DCMAKE_BUILD_TYPE=${BUILD_TYPE}
-D${PROJECT_NAME}_BUILD_ONE_CONFIG=ON
)
endmacro()
setup(windows_x86 Release)
setup(windows_x86 Debug)
setup(windows_x86_64 Release)
setup(windows_x86_64 Debug)
setup(linux_x86_64 Release)
setup(linux_x86_64 Debug)
setup(linux_aarch64 Release)
setup(linux_aarch64 Debug)
endif()
But I have 2 problems with the ExternalProject approach with the Ninja generator that I think are directly related to each other.
-
When the top level project is run, I'm shown on the command prompt a series of steps that Ninja is running, which amounts to downloading (not needed in my case), patching (also not needed), configuring, building, installing. However, at the very least the building step would normally have several thousand sub-steps that are being run, and I don't see any of that at all. Instead, I get a dump of the results of each step only when they finish in entirety. For multi-thousand source file projects, this takes up to several hours, which is a problem for interactive change/compile/debug loops where the developer might realize something is very wrong with the build based on observing the warnings from the first handful of files being compiled.
-
By default, ninja uses parallelism equal to the current machine's number of processor cores. If you're building 8 different configurations like I am, this gives you ${numproc}*8 parallel tasks. This causes absolute havok on disk access, as well as CPU load.
When using the default BUILD_COMMAND (e.g. not providing this parameter at all), I would have expected that when the top-level generator and the external project generator are both using Ninja, that only one Ninja process is needed, and that the top-level ninja would include the external project's ninja files as subninjas. In this way, the individual build commands get interleaved properly on the command line output, as well as ninja being able to properly schedule parallelism.