Update DEPFILE of add_custom_command at build time
CMake 3.7 introduces DEPFILE option for add_custom_command:
Specify a depfile which holds dependencies for the custom command. It is usually emitted by the custom command itself.
However, it seems that CMake reads the depfile before it is generated, and executes custom commands in an order as outdated depfile.
Here's a demo:
CMakeLists.txt
cmake_minimum_required(VERSION 3.25)
project(test)
if(NOT DEFINED n)
set(n 3)
endif()
message("n = ${n}")
add_custom_command(
OUTPUT depfile
COMMAND cp ${CMAKE_CURRENT_SOURCE_DIR}/src_depfile depfile
COMMAND date > check
DEPENDS src_depfile
VERBATIM)
foreach(i RANGE 1 ${n})
add_custom_command(
OUTPUT ${i}
COMMAND python3 ${CMAKE_CURRENT_SOURCE_DIR}/run.py ${i}
DEPENDS depfile
DEPFILE depfile
VERBATIM)
list(APPEND lst ${i})
endforeach()
add_custom_target(test ALL DEPENDS ${lst})
src_depfile: 2 depends on 1 and 3
2: 1 3
run.py
import sys, os, time
tgt = sys.argv[1]
deps = next(filter(lambda s: s.startswith(tgt + ':'),
open('depfile').read().splitlines()),
None)
check = open('check').read()
if deps:
for dep in deps.split(':')[1].split():
if not os.path.exists(dep):
print(f'Error: file "{dep}" does not exist.', file=sys.stderr)
exit(1)
if open(dep).read() != check:
print(f'Error: file "{dep}" is outdated.', file=sys.stderr)
exit(1)
time.sleep(1)
open(tgt, 'w').write(check)
When doing a totally fresh build, 2 is being built before 3 is complete,
despite that depfile is indeed generated before other commands.
$ rm -rf build
$ cmake -S . -B build
$ cmake --build build
[ 25%] Generating depfile
[ 50%] Generating 1
[ 75%] Generating 2
Error: file "3" does not exist.
make[2]: *** [2] Error 1
make[1]: *** [CMakeFiles/test.dir/all] Error 2
make: *** [all] Error 2
The build succeeds when we re-build immediately, without changing src_depfile.
This time, 3 is built before 2 because depfile(that is generated in last failed build) is up to date.
$ cmake --build build
[ 25%] Generating 3
[ 50%] Generating 2
[100%] Built target test
However, then we modify src_depfile so that 1 depends on 3, the build fails again.
1: 3
2: 1 3
This time, it fails because we check if all dependencies is up to date. If we don't, it will continue with outdated(or worse, corrupted, if it is being built at the same time) dependencies.
$ cmake --build build
[ 25%] Generating depfile
[ 50%] Generating 1
Error: file "3" is outdated.
make[2]: *** [1] Error 1
make[1]: *** [CMakeFiles/test.dir/all] Error 2
make: *** [all] Error 2
Is there a way to fix or avoid this problem when updating depfile at build time?