Skip to content

Update DEPFILE of add_custom_command at build time

CMake 3.7 introduces DEPFILE option for add_custom_command:

Specify a depfile which holds dependencies for the custom command. It is usually emitted by the custom command itself.

However, it seems that CMake reads the depfile before it is generated, and executes custom commands in an order as outdated depfile.

Here's a demo:

CMakeLists.txt

cmake_minimum_required(VERSION 3.25)

project(test)
 
if(NOT DEFINED n)
  set(n 3)
endif()

message("n = ${n}")

add_custom_command(
  OUTPUT depfile
  COMMAND cp ${CMAKE_CURRENT_SOURCE_DIR}/src_depfile depfile
  COMMAND date > check
  DEPENDS src_depfile
  VERBATIM)

foreach(i RANGE 1 ${n})
  add_custom_command(
    OUTPUT ${i}
    COMMAND python3 ${CMAKE_CURRENT_SOURCE_DIR}/run.py ${i}
    DEPENDS depfile
    DEPFILE depfile
    VERBATIM)
  list(APPEND lst ${i})
endforeach()

add_custom_target(test ALL DEPENDS ${lst})

src_depfile: 2 depends on 1 and 3

2: 1 3

run.py

import sys, os, time
tgt = sys.argv[1]
deps = next(filter(lambda s: s.startswith(tgt + ':'),
                   open('depfile').read().splitlines()),
            None)
check = open('check').read()
if deps:
    for dep in deps.split(':')[1].split():
        if not os.path.exists(dep):
            print(f'Error: file "{dep}" does not exist.', file=sys.stderr)
            exit(1)
        if open(dep).read() != check:
            print(f'Error: file "{dep}" is outdated.', file=sys.stderr)
            exit(1)
time.sleep(1)
open(tgt, 'w').write(check)

When doing a totally fresh build, 2 is being built before 3 is complete, despite that depfile is indeed generated before other commands.

$ rm -rf build
$ cmake -S . -B build
$ cmake --build build
[ 25%] Generating depfile
[ 50%] Generating 1
[ 75%] Generating 2
Error: file "3" does not exist.
make[2]: *** [2] Error 1
make[1]: *** [CMakeFiles/test.dir/all] Error 2
make: *** [all] Error 2

The build succeeds when we re-build immediately, without changing src_depfile. This time, 3 is built before 2 because depfile(that is generated in last failed build) is up to date.

$ cmake --build build
[ 25%] Generating 3
[ 50%] Generating 2
[100%] Built target test

However, then we modify src_depfile so that 1 depends on 3, the build fails again.

1: 3
2: 1 3

This time, it fails because we check if all dependencies is up to date. If we don't, it will continue with outdated(or worse, corrupted, if it is being built at the same time) dependencies.

$ cmake --build build
[ 25%] Generating depfile
[ 50%] Generating 1
Error: file "3" is outdated.
make[2]: *** [1] Error 1
make[1]: *** [CMakeFiles/test.dir/all] Error 2
make: *** [all] Error 2

Is there a way to fix or avoid this problem when updating depfile at build time?

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information