Ninja Multi-Config: Incorrect order-only dependencies for library targets when using AUTOMOC depfile support
Wall of text incoming.
While working on !4289 (merged) I found a bug regarding incorrect Mult Config-Ninja rule dependencies.
Unfortunately it's a bit hard to grasp what's going on due to many different factors working together.
Sample project requiring CMake 3.17
cmake_minimum_required(VERSION 3.16.0)
if(CMAKE_GENERATOR STREQUAL "Ninja Multi-Config")
set(CMAKE_NMC_DEFAULT_BUILD_FILE_CONFIG "Release" CACHE STRING "")
set(CMAKE_NMC_CROSS_CONFIGS "all" CACHE STRING "")
set(CMAKE_NMC_DEFAULT_CONFIGS "all" CACHE STRING "")
endif()
#set(USE_FRAMEWORKS ON)
if(USE_FRAMEWORKS AND POLICY CMP0103)
# Important to allow creation of debug and release frameworks with Ninja Multi-Config
cmake_policy(SET CMP0103 NEW)
endif()
project(multi_ninja_cross_config_dependencies LANGUAGES CXX)
# Make a source file
set(source_name "source.cpp")
set(source_path "${CMAKE_CURRENT_BINARY_DIR}/${source_name}")
file(WRITE "${source_path}" "void foo() {}")
# Important not to create implicit dependencies in the Ninja library rule.
set(CMAKE_LINK_DEPENDS_NO_SHARED ON)
# Differentiate debug libraries / frameworks.
set(CMAKE_DEBUG_POSTFIX "_debug")
# Important to trigger the new Ninja depfile >5.15 code branch in cmQtAutoGenInitializer::InitAutogenTarget
# Actual Qt not needed.
set(QT_VERSION_MAJOR 6)
set(fake_moc_path "${CMAKE_CURRENT_BINARY_DIR}/fake_moc")
file(WRITE "${fake_moc_path}" "")
# Core library
add_library(DummyCore SHARED "${source_path}")
set_target_properties(DummyCore PROPERTIES AUTOMOC ON AUTOMOC_EXECUTABLE "${fake_moc_path}")
if(USE_FRAMEWORKS)
set_target_properties(DummyCore PROPERTIES FRAMEWORK ON)
endif()
# Gui Library
add_library(DummyGui SHARED "${source_path}")
target_link_libraries(DummyGui PUBLIC DummyCore)
set_target_properties(DummyGui PROPERTIES AUTOMOC ON AUTOMOC_EXECUTABLE "${fake_moc_path}")
if(USE_FRAMEWORKS)
set_target_properties(DummyGui PROPERTIES FRAMEWORK ON)
endif()
# For debugging
set_property(GLOBAL PROPERTY GLOBAL_DEPENDS_DEBUG_MODE ON)
Configure and build with
$ cmake .. -G"Ninja Multi-Config" -DCMAKE_CONFIGURATION_TYPES="Release;Debug" -DUSE_FRAMEWORKS=ON
$ ninja -f build-Release.ninja DummyGui:Debug
Result
Linking CXX shared library Debug/DummyGui.framework/Versions/A/DummyGui_debug
FAILED: Debug/DummyGui.framework/Versions/A/DummyGui_debug
clang: error: no such file or directory: 'Debug/DummyCore.framework/Versions/A/DummyCore_debug'
$ ninja -t browse
target is built using rule CXX_SHARED_LIBRARY_LINKER__DummyGui_Debug of
CMakeFiles/DummyGui.dir/Debug/DummyGui_autogen/mocs_compilation.cpp.o
CMakeFiles/DummyGui.dir/Debug/source.cpp.o
DummyGui_autogen:Release (order-only)
Release/DummyCore.framework/Versions/A/DummyCore (order-only) # <--------------
Note that the Debug library depends on Release/DummyCore.framework/Versions/A/DummyCore
the Release
library variant.
If I comment out set(QT_VERSION_MAJOR 6)
then the browse
output is correct (dependency on Debug variant) and the build succeeds.
target is built using rule CXX_SHARED_LIBRARY_LINKER__DummyGui_Debug of
CMakeFiles/DummyGui.dir/Debug/DummyGui_autogen/mocs_compilation.cpp.o
CMakeFiles/DummyGui.dir/Debug/source.cpp.o
Debug/DummyCore.framework/Versions/A/DummyCore_debug (order-only) # <--------------
DummyGui_autogen:Release (order-only)
I initially thought maybe my implementation of the CMP01030
feature is incomplete, but if you configure without frameworks, the same dependency weirdness happens.
Although the build does not fail due to the build graph having an additional correct order-only dependency (I haven't tracked down where this one comes from yet).
For comparison without frameworks:
# Without set(QT_VERSION_MAJOR 6) (correct dependencies)
target is built using rule CXX_SHARED_LIBRARY_LINKER__DummyGui_Debug of
CMakeFiles/DummyGui.dir/Debug/DummyGui_autogen/mocs_compilation.cpp.o
CMakeFiles/DummyGui.dir/Debug/source.cpp.o
Debug/libDummyCore_debug.dylib (order-only) # <--------------
Debug/libDummyCore_debug.dylib (order-only) # <--------------
DummyGui_autogen:Release (order-only)
# With set(QT_VERSION_MAJOR 6) (strange dependencies)
target is built using rule CXX_SHARED_LIBRARY_LINKER__DummyGui_Debug of
CMakeFiles/DummyGui.dir/Debug/DummyGui_autogen/mocs_compilation.cpp.o
CMakeFiles/DummyGui.dir/Debug/source.cpp.o
Debug/libDummyCore_debug.dylib (order-only) # <--------------
DummyGui_autogen:Release (order-only)
Release/libDummyCore.dylib (order-only) # <---------- why is this here?
While investigating the behavior I tracked it down to the following flow:
- If Qt version > 5.15 (which has the new ninja depfile feature), then
cmQtAutoGenInitializer::InitAutogenTarget
adds a custom_command with a target dependency onDummyCore
forDummyGui
. - This ends up creating an utility target called
DummyCore
viacmTargetTraceDependencies::FollowCommandDepends
->cmTargetTraceDependencies::IsUtility
- During Ninja file generation,
cmNinjaNormalTargetGenerator::WriteLinkStatement
->cmGlobalNinjaGenerator::AppendTargetDepends
ends up doing the following check
if (targetDep.IsUtil() &&
targetDep->GetType() != cmStateEnums::OBJECT_LIBRARY) {
this->AppendTargetOutputs(targetDep, outs, fileConfig, depends);
} else {
this->AppendTargetOutputs(targetDep, outs, config, depends);
}
isUtil
is true for the created DummyCore
utility target, so the first code branch is taken. The value of fileConfig
is Release
while generating the build-Release.ninja
file, but the actual current target config is Debug
, which ends up creating the wrong order only dependency.
In constrast if QT_VERSION_MAJOR
is commented out, no add_custom_command is created, there is no CoreDummy
utility target, and the second branch is taken, where the correct config
is selected rather than fileConfig
.
This seems strange. When generating a cross configuration in build-Release.ninja
for a Debug
target, why would config
or fileConfig
be chosen dependent on the target type?
I'll have to dig a bit more on why for the non-framework case a correct order only dependency is added.
And why for the framework case the Utility DummyCore
target somehow seems to eat the non-Utility DummyCore
target which is still present, as shown in the output of GLOBAL_DEPENDS_DEBUG_MODE
:
The final target dependency graph is:
target 0 is [rebuild_cache]
target 1 is [edit_cache]
target 2 is [DummyGui]
depends on target 3 [DummyCore] (weak) # <-------
depends on target 3 [DummyCore] (strong) # <------- the one added by add_custom_command when QT_VERSION_MAJOR == 6
depends on target 4 [DummyGui_autogen] (strong)
target 3 is [DummyCore]
depends on target 5 [DummyCore_autogen] (strong)
target 4 is [DummyGui_autogen]
depends on target 3 [DummyCore] (strong)
target 5 is [DummyCore_autogen]