Ninja: Implicit directories can't be parsed if there are ANSI colors in NINJA_STATUS
This is an issue that pertains to parsing implicit directories but I think that it's also showing problems in other areas that, eventually, cause the parsing to fail.
While investigating "Cannot generate a safe runtime search path for target" messages that most developers were having, except for a few that were not, I found that the the safe runtime
message showed up only when implicit linker paths had been parsed. Doing a diff between --trace-expand
output between two machines I saw that lines were being ignored when parsing implicit link information when text was shown in colors in CMakeFiles/CMakeOutput.log
as shown in the following screen capture:
Whenever there are ANSI codes it seems like CMake can't convert a string to a list. In CMakeParseImplicitLinkInfo.cmake
around line 32 there is code to replace newlines with a semicolon so the foreach
command can iterate over each line:
string(APPEND log " link line regex: [${linker_regex}]\n")
string(REGEX REPLACE "\r?\n" ";" output_lines "${text}")
foreach(line IN LISTS output_lines)
if ${text}
has ANSI codes the semicolons do not act as list separators (maybe the string is being considered binary?). As a result, that long line doesn't match the parsing regexp and it's ignored:
ignore line: [Run Build Command(s):/usr/local/bin/ninja cmTC_1f101 && [0.014|1/1/1] Building C object CMakeFiles/cmTC_1f101.
If I add code to CMakeParseImplicitLinkInfo.cmake
to remove ANSI code, ${text}
is succesfully converted to a list and parsing can proceed:
string(APPEND log " link line regex: [${linker_regex}]\n")
string(ASCII 27 Esc)
string(REGEX REPLACE "${Esc}9B|${Esc}\[)[0-?]*[ -\/]*[@-~]" "" text "${text}")
string(REGEX REPLACE "\r?\n" ";" output_lines "${text}")
foreach(line IN LISTS output_lines)
How did ANSI text got there? Ninja allows users to customize its status text via the NINJA_STATUS
variable, and some users where colorizing it:
export NINJA_STATUS="$(echo -e "\e[1;40m\e[37m[%e|\e[0;40;92m%u/\e[93m%r\e[92m/%f\e[93m\e[37m]\e[39m\e[0;49m ")"
My theory is that Ninja's status is rendered as-is and no filtering of ANSI sequences is done when output is redirected to a non-tty. All other text is correctly stripped of ansi sequences. I test by building wih ninja | less
which only shows the prompt in colors, while compiler errors or npm
installs that we do when building are correctly monochrome.
My points here are:
- There is a Ninja bug with
NINJA_STATUS
- Why a semicolon separated string is not treated as a list when ANSI codes are present?
- Should CMake protect itself and strip ANSI codes when parsing compiler/tools output? Maybe a switch like
-fdiagnostic-colors=always
could be in use and that might break parsing too (I haven't tested that).
Steps to reproduce the problem:
- Create a simple CMakeLists.txt file:
cmake_minimum_required(VERSION 3.10)
project(test)
- Configure the project with the Ninja generator
cmake -G Ninja ..
and look atCMakeFiles/CMakeOutput.log
. It shows success:
Parsed C implicit link information from above output:
link line regex: [^( *|.*[/\])(ld|CMAKE_LINK_STARTFILE-NOTFOUND|([^/\]+-)?ld|collect2)[^/\]*( |$)]
ignore line: [Change Dir: /home/pnavarro/test/cm/ansi/a/CMakeFiles/CMakeTmp]
ignore line: []
ignore line: [Run Build Command(s):/usr/local/bin/ninja cmTC_62456 && [1/2] Building C object CMakeFiles/cmTC_62456.dir/CMakeCCompilerABI.c.o]
ignore line: [Using built-in specs.]
ignore line: [COLLECT_GCC=/usr/bin/cc]
ignore line: [OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa]
ignore line: [OFFLOAD_TARGET_DEFAULT=1]
ignore line: [Target: x86_64-linux-gnu]
(...)
arg [/usr/lib/gcc/x86_64-linux-gnu/11/collect2] ==> ignore
arg [-plugin] ==> ignore
arg [/usr/lib/gcc/x86_64-linux-gnu/11/liblto_plugin.so] ==> ignore
arg [-plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/11/lto-wrapper] ==> ignore
arg [-plugin-opt=-fresolution=/tmp/ccIp04iN.res] ==> ignore
- Colorize
NINJA_STATUS
export NINJA_STATUS="$(echo -e "\e[1;40m\e[37m[%e|\e[0;40;92m%u/\e[93m%r\e[92m/%f\e[93m\e[37m]\e[39m\e[0;49m ")"
- Configure again with the Ninja generator
cmake -G Ninja ..
and look atCMakeFiles/CMakeOutput.log
. Parsing failed (lines truncated for brevity):
Parsed C implicit link information from above output:
link line regex: [^( *|.*[/\])(ld|CMAKE_LINK_STARTFILE-NOTFOUND|([^/\]+-)?ld|collect2)[^/\]*( |$)]
ignore line: [Change Dir: /home/pnavarro/test/cm/ansi/a/CMakeFiles/CMakeTmp]
ignore line: []
ignore line: [Run Build Command(s):/usr/local/bin/ninja cmTC_48b44 && [0.012|1/1/1] Building C object CMakeFiles/cmTC_48b44.dir/CMakeCCompilerABI.c.o;Using built-in specs.;COLLECT_GCC=/usr/bin/cc;OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa;OFFLOAD_TARGET_DEFAULT=1;Target: x (...)
(...)
implicit libs: []
implicit objs: []
implicit dirs: []
implicit fwks: []