IPO: Linking with GCC and Clang would require the used arguments for compiling
See the section about -flto from the GCC Optimize Options.
There are heuristics that seem to kinda work on common OS's, but they do only partially work when it comes to embedded controller. For ARM, you have several options affection binary compatibility (fpu, vector-extensions,...), not replicating them during linking will give you an error.
In a concrete projects, these C/C++ flags where atleast required for the linker: -mcpu=cortex-m0 -mthumb -mfloat-abi=soft -fshort-enums
Other might work or create subtle issues, quite possibly different depending on the compilers heuristics.
To make this more reliable, the CMAKE_*_LINK_FLAGS
for GCC/Clang should have some placeholder <*_COMPILE_ARGS>
which expands to the same arguments (flags, includes, defines) like when compiling.