Querying GENERATED source file property tricks CMake into searching the binary directory for non-GENERATED files
Hello everyone,
I've encountered some unexpected behaviour (which I assume might be a bug) when calling get_source_file_property(_var _file GENERATED)
on a file that is not a generated one. Querying the GENERATED
property tricks CMake into searching the binary directory for this file at generation time (while the expected behaviour is to search the source directory for the file). Please see this github repo for a small example showcasing the issue/for a small example to reproduce the behaviour.
Update: Even though the issue remains and the background information is still valid, I managed to narrow the example to reproduce the behaviour. Please see my comment down below for a way simpler example. Sorry for the noise
To avoid the XY problem, some background: I'm currently working on a custom CMake-integration for Doxygen. I try to populate the input to Doxygen (i.e. source files and include directories) automatically from information that is already known to CMake. To automatically collect source files, I query a target's SOURCES property and try to parse its content similar to what is discussed in the CMake discourse.
One thing I try to achieve here is to convert relative paths to absolute ones before passing the paths to Doxygen. To this end, I'm currently using the following code (it does not cover all input correctly, but it suffices to showcase the overall issue):
function(absolutify_source var source target)
list(APPEND CMAKE_MESSAGE_CONTEXT "absolutify_source")
get_target_property(_target_source_dir ${target} SOURCE_DIR)
get_target_property(_target_binary_dir ${target} BINARY_DIR)
# There are several possibilities of what we get as input in `source`.
# See https://cmake.org/cmake/help/latest/prop_tgt/SOURCES.html for details.
# All in all, we employ the following steps:
# 1. If `source` contains a genex, leave it as is. Genexes are expected to evaluate to absolute paths.
# 2. If `source` is an absolute path, leave it as is.
# 3. For everything else (i.e. relative paths):
# 3.1 Check for the file being known to CMake in the binary dir. If it is known and it has the GENERATED
# property set, this file takes precedence. Return the absolute path to the
# found file (relative paths for generated files are always considered to be relative to the binary dir).
# If not, continue with 3.2.
# 3.2 Check for the file being present in the source dir and the binary dir (in this
# order). Return the absolute path to the first location that points to an existing file.
# The GENERATED property does not matter.
set(_source_abs)
# check for genexes - contains_genex is a custom function
contains_genex(_has_genex "${source}")
if(_has_genex)
# there is a genex in source, according to the documentation we
# may assume it evaluates to an absolute path
# (see https://cmake.org/cmake/help/latest/prop_tgt/SOURCES.html)
set(_source_abs "${source}")
else()
# no genex in source, path can be both relative and absolute
cmake_path(IS_ABSOLUTE source _is_absolute)
if(_is_absolute)
# source is absolute path, nothing to do except normalization
cmake_path(NORMAL_PATH source OUTPUT_VARIABLE _source_abs)
else()
message(WARNING "Relative path detected: ${source}. Consider using absolute paths to silence this warning. For basic use-cases it is sufficient to prepend \"\${CMAKE_CURRENT_SOURCE_DIR}/\".")
# Check for a generated file
# Note: the GENERATED property has some known unexpected behaviours and maybe even bugs.
# See the following for a discussion:
#
# - https://discourse.cmake.org/t/unexpected-behavior-of-the-generated-source-file-property-and-cmp0118/3821/3
# - https://discourse.cmake.org/t/behavior-of-where-cmp0118s-value-is-used-is-ambiguous/4045
# - https://gitlab.kitware.com/cmake/cmake/-/issues/18399
#
# All in all, the current way to query the GENERATED property is to use `get_source_file_property` with
# - the absolute path the generated file (the file does not have to exist on disc yet)
# - the absolute path to the directory in which the file is introduced to CMake (i.e. the directory in
# which the introducing `CMakeLists.txt` lives - out of convenience, we assume this directory is equal to
# the TARGET_DIRECTORY of TARGET).
cmake_path(ABSOLUTE_PATH source BASE_DIRECTORY "${_target_binary_dir}" NORMALIZE OUTPUT_VARIABLE _generated_candidate)
get_source_file_property(_generated "${_generated_candidate}" TARGET_DIRECTORY "${target}" GENERATED)
message(DEBUG "Processing ${source}")
message(DEBUG " candidate location for generated file: ${_generated_candidate}")
if(_generated)
message(DEBUG " File is GENERATED")
set(_source_abs "${_generated_candidate}")
else()
message(DEBUG " File is NOT GENERATED.")
# the path can be relative both to the target's source and binary directory,
# hence we use `find_file` to search for the file
cmake_path(GET source PARENT_PATH _source_relative)
cmake_path(GET source FILENAME _source_filename)
cmake_path(ABSOLUTE_PATH _source_relative BASE_DIRECTORY "${_target_source_dir}" NORMALIZE OUTPUT_VARIABLE _source_source_dir)
cmake_path(ABSOLUTE_PATH _source_relative BASE_DIRECTORY "${_target_binary_dir}" NORMALIZE OUTPUT_VARIABLE _source_binary_dir)
find_file(
_source_abs
"${_source_filename}"
PATHS "${_source_source_dir}" "${_source_binary_dir}"
NO_CACHE
REQUIRED
NO_DEFAULT_PATH
)
endif()
endif()
endif()
set(${var} "${_source_abs}" PARENT_SCOPE)
endfunction()
Now on the issue I'm encountering: when given a relative path in source
, I check whether it refers to a generated file (such file will be located in the binary directory instead of the source directory from how I read the docs on SOURCES
). To this end, I call get_source_file_property
on some candidate path _generated_candidate
. No matter what the returned value is, CMake will search for source
relative to the binary directory at generation time afterwards. This causes the generation to fail for non-generated files (which live somewhere in the source directory). If I replace the call to get_source_file_property
by some hard-coded logic (again, please see the above github repo; there I have implemented an option to switch between the hardcoded workaround and the intended call to get_source_file_property
- i.e. toggle the option TRIGGER_WRONG_SEARCH
), everything works as expected.
Here's the output when configuring the above github repo on my machine:
> cd /some/path/cmake_generated_bug/build
> cmake -B . -S .. --log-context --log-level DEBUG -DTRIGGER_WRONG_SEARCH:BOOL=OFF
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning at cmake/AbsPath.cmake:84 (message):
Relative path detected: bar.cpp. Consider using absolute paths to silence
this warning. For basic use-cases it is sufficient to prepend
"${CMAKE_CURRENT_SOURCE_DIR}/".
Call Stack (most recent call first):
src/CMakeLists.txt:12 (absolutify_source)
-- [absolutify_source] Processing bar.cpp
-- [absolutify_source] candidate location for generated file: /some/path/cmake_generated_bug/build/src/bar.cpp
-- [absolutify_source] File is NOT GENERATED.
-- Absolute path to bar.cpp: /some/path/cmake_generated_bug/src/bar.cpp
CMake Warning at cmake/AbsPath.cmake:84 (message):
Relative path detected: bar.h. Consider using absolute paths to silence
this warning. For basic use-cases it is sufficient to prepend
"${CMAKE_CURRENT_SOURCE_DIR}/".
Call Stack (most recent call first):
src/CMakeLists.txt:15 (absolutify_source)
-- [absolutify_source] Processing bar.h
-- [absolutify_source] candidate location for generated file: /some/path/cmake_generated_bug/build/src/bar.h
-- [absolutify_source] File is GENERATED
-- Absolute path to bar.h: /some/path/cmake_generated_bug/build/src/bar.h
-- Configuring done
-- Generating done
-- Build files have been written to: /some/path/cmake_generated_bug/build
> rm -r *
> cmake -B . -S .. --log-context --log-level DEBUG -DTRIGGER_WRONG_SEARCH:BOOL=ON
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning at cmake/AbsPath.cmake:84 (message):
Relative path detected: bar.cpp. Consider using absolute paths to silence
this warning. For basic use-cases it is sufficient to prepend
"${CMAKE_CURRENT_SOURCE_DIR}/".
Call Stack (most recent call first):
src/CMakeLists.txt:12 (absolutify_source)
-- [absolutify_source] Processing bar.cpp
-- [absolutify_source] candidate location for generated file: /some/path/cmake_generated_bug/build/src/bar.cpp
-- [absolutify_source] File is NOT GENERATED.
-- Absolute path to bar.cpp: /some/path/cmake_generated_bug/src/bar.cpp
CMake Warning at cmake/AbsPath.cmake:84 (message):
Relative path detected: bar.h. Consider using absolute paths to silence
this warning. For basic use-cases it is sufficient to prepend
"${CMAKE_CURRENT_SOURCE_DIR}/".
Call Stack (most recent call first):
src/CMakeLists.txt:15 (absolutify_source)
-- [absolutify_source] Processing bar.h
-- [absolutify_source] candidate location for generated file: /some/path/cmake_generated_bug/build/src/bar.h
-- [absolutify_source] File is GENERATED
-- Absolute path to bar.h: /some/path/cmake_generated_bug/build/src/bar.h
-- Configuring done
CMake Error at src/CMakeLists.txt:6 (add_library):
Cannot find source file:
/some/path/cmake_generated_bug/build/src/bar.cpp
CMake Error at src/CMakeLists.txt:6 (add_library):
No SOURCES given to target: foo
CMake Generate step failed. Build files cannot be regenerated correctly.
Calling get_source_file_property
should not cause CMake to expect a given file somewhere else, right? After all, it's a getter - not a setter ;)
Sorry for the lengthly report, I tried to condense the example down to its bare minimum. But this is the closest I was able to get with regard to "minimality".