# Configure-time feature checks: batching many `try_compile`-class checks into few compiler invocations ## Problem Configure time on check-heavy projects is dominated by the *number* of feature checks, and each classic check costs far more than the question it answers. `check_symbol_exists("printf" ...)` does not just compile three lines — it spawns a full nested CMake: configure a throwaway one-target project, generate it, then invoke the build tool. The scaffolding dwarfs the actual compile, and the checks run strictly sequentially. CMake's own tree is a representative victim. Profiling a fresh configure with CMake's built-in profiler (`--profiling-format=google-trace`, self/exclusive time, so a nested build counts against the command that spawned it): | Run | `try_compile` invocations | Configure time spent in checks | |---|---|---| | stock CMake, cold configure | **457** | **~95%** | Nearly all of a cold configure of CMake is feature checking. Any project doing autoconf-style detection (curl, libarchive, liblzma, kwsys — all bundled here) pays a version of this. ## Root observation The checks split into two very different classes, and only one of them actually needs a nested build: 1. **Introspectable** — answerable by the preprocessor/compiler *without the TU failing*: `__has_include`, `__has_builtin`, `defined()`, `sizeof`. The "no" answer is a **value**, not an error. Many checks are in this class. 2. **Failure-is-the-answer** — the "no" outcome is a compile or link failure (symbol/member existence, call signatures, link resolution). These must be isolated so one failure doesn't poison anything else. Class 1 can all share **one** always-successful translation unit. Class 2 still needs a compile/link each, but *not* a nested configure each — they can be targets in a single keep-going, parallel build. ## Approach A `CheckBatch` module: declare a batch of queries, run it once, read results from the usual cache variables. - **Survey compile** — all class-1 answers for a language are emitted into one always-successful TU, each encoded as a magic string baked into the object file (the technique `CheckTypeSize` already uses for sizes, generalized) and recovered with `file(STRINGS)`. One `try_compile` answers any number of them. Nothing executes → cross-compile safe. - **Probe build** — class-2 queries each become a tiny object library or executable in one generated sub-project, built with `--parallel` and keep-going (`ninja -k 0` / `make -k`). A query succeeded iff its artifact exists. One nested configure+build, amortized over every probe in the batch. ```cmake include(CheckBatch) check_batch(BEGIN posix LANGUAGE C HEADERS stdio.h stdlib.h) check_batch(ADD posix INCLUDE HAVE_SYS_EPOLL_H sys/epoll.h) check_batch(ADD posix TYPE_SIZE SIZEOF_OFF_T off_t) check_batch(ADD posix SYMBOL HAVE_GETADDRINFO getaddrinfo HEADERS netdb.h) check_batch(ADD posix FUNCTION HAVE_EVENTFD eventfd) check_batch(RUN posix) if(HAVE_GETADDRINFO) # ordinary cache variables ``` **Design constraints held throughout:** - **Opt-in and additive.** No existing module changes. Projects migrate their check calls incrementally, block by block. - **Result-compatible.** Results are published as the same `INTERNAL` cache variables/helpstrings the classic modules produce, so generated config headers, `#ifdef`s and `HAVE_*` consumers are untouched. - **Transparent fallback.** Batching engages only when the compiler supports `__has_include` and the probe generator supports keep-going builds — decided from *published* compiler facts (GNU ≥ 5, Clang/AppleClang/IntelLLVM, MSVC ≥ 19.11, or `cxx_std_17`/`c_std_23` in `COMPILE_FEATURES`), with a cached probe only for unknown compilers. Otherwise every query runs through the classic `Check*` modules with identical results. - **Self-verifying.** `CHECKBATCH_VERIFY=ON` re-runs every batch through the classic modules into shadow variables and warns on divergence — a migration safety net. ## Results Prototype exists. You have an implementation in [this MR](https://gitlab.kitware.com/cmake/cmake/-/merge_requests/12269) and a branch with CMake updates to use this [here](https://gitlab.kitware.com/Orphis/cmake/-/commits/checkbatch-demo): the module, an adoption of it in the bundled `cmcurl`/`cmlibarchive`/`cmliblzma`/`kwsys` checks, and a dependency-free cross-platform benchmark driver (`cmake -P`). Effect on the profile (same workload as above): | Run | `try_compile` invocations | Time in checks | |---|---|---| | classic fallback | 457 | ~95% | | batched (cold) | **59** | ~89% | | batched (re-run) | **0** | — | Wall clock, Windows 11 / 32 cores. **Synthetic** workload = ~90 independent checks (isolates the feature); **cmake** = configuring this tree (whole configure, so the speedup is diluted by everything else configure does): | Workload | Environment | Classic | Batched | Speedup | Cached re-run | |---|---|---|---|---|---| | synthetic | Ninja + Clang | 86.7 s | 9.6 s | ~9× | 0.33 s | | synthetic | VS generator + `CHECKBATCH_GENERATOR=Ninja` | 188.0 s | 12.2 s | ~15× | 0.28 s | | cmake | Ninja + Clang | 67.1 s | 44.3 s | ~1.5× | 3.25 s | | cmake | VS generator + `CHECKBATCH_GENERATOR=Ninja` | 97.0 s | 61.6 s | ~1.6× | 8.7 s | Correctness: generated config headers are **byte-identical** to the unmodified tree across all migrated libraries, and `CHECKBATCH_VERIFY` reports **zero divergences**. The Visual Studio rows are the sharpest illustration of a support generator: under the VS generator each classic check is a full MSBuild invocation (188 s for ~90 checks), which a Ninja-backed probe build collapses into one parallel build — while the user's build stays a VS project. **Availability:** the module needs only long-established commands (newest is `try_compile(SOURCE_FROM_VAR)`, CMake 3.25), so projects can vendor it locally today and drop the copy once their minimum CMake ships it. ## Where the remaining cost is Classifying the 349 probe-kind checks across the four migrated libraries: | Bucket | Count | Nature | |---|---|---| | **A** — constant-expression / preprocessor-detectable | 94 | could be *free* in the survey but aren't yet | | **B** — hard-fail compile | 92 | "no" is an intended compile error in C | | **C** — link-time | 163 | linker must resolve a symbol — not compile-answerable | Bucket A is 94 checks currently paying a probe compile for an answer the preprocessor already knows — 86 of them are libarchive's glibc `__stub_*` tests (`#if defined __stub_foo || defined __stub___foo`). ## Roadmap / design space Grouped by status. Happy to reorder based on what the team considers valuable. ### Implemented in the prototype - [x] Survey + probe execution, kinds: `INCLUDE`, `MACRO`, `BUILTIN`, `TYPE_SIZE`, `SYMBOL`, `COMPILES`, `FUNCTION`, `PROGRAM` - [x] Transparent classic fallback + capability gate from published compiler facts - [x] `CHECKBATCH_VERIFY` cross-check harness - [x] `CHECKBATCH_GENERATOR` — probe builds under Ninja while the main build is an IDE generator (Visual Studio / Xcode) - [x] `CHECKBATCH_JOBS`, `CHECKBATCH_QUIET` (+ per-batch `QUIET`), `CheckBatchError.log` - [x] Mixed-language batches (per-entry `LANGUAGE`; one survey TU per language) - [x] Per-entry `HEADERS`/`INCLUDES` for `TYPE_SIZE` ### Designed, not yet implemented - [ ] **`PP` kind — preprocessor `#if` expression survey.** Language-agnostic, no modern features needed. Harvests the 94 bucket-A checks (the 86 glibc stub tests, bswap, IOC/errno macros, `HAVE_TIME_T_UNSIGNED`, `HAVE_FILE_OFFSET_BITS`). Biggest ROI-per-effort. - [ ] **Transparent C++ upgrade.** When a batch is C++ on a capable compiler, resolve member/type/call/expression checks in the *survey* via `requires`-expressions (C++20) or the detection idiom (C++17), instead of a probe. Requires headers as inputs **and** the check expressed as a dependent-operand call/expression — then an undeclared name becomes a substitution failure (soft `false`), not a hard error. Would make libarchive's struct-member family and curl's call-signature ladders (`gethostbyname_r_{3,5,6}`, `ioctl*`, `fsetxattr_{5,6}`) free *for C++ consumers*. New structured kinds: `MEMBER`, `CALL`, `EXPRESSION`. **Boundary:** the bare address-of `check_symbol_exists` form (unknown signature) cannot be made soft — there's no call to carry the dependence, and naming a non-dependent undeclared identifier is a hard error. It upgrades only if re-expressed as `CALL` with argument types. - [ ] **`COMPILE_FLAG` / `LINK_FLAG`.** Flag checks can't be surveyed (a flag is a property of the invocation, and support is decided by scanning output for an unrecognized-flag diagnostic), but they fold into the batch's *existing* probe build as targets carrying `target_compile_options`/`target_link_options` — removing the per-flag nested configure entirely. - [ ] **`LIBRARY`** (`CheckLibraryExists`), **`VARIABLE`** (`CheckVariableExists`), **`PROTOTYPE`** (`CheckPrototypeDefinition`), **`RUNS`** (`check_source_runs`/`try_run`: compile in the probe build, then execute survivors; non-cross only). - [ ] **Shared persistent probe project.** Reuse one probe workspace per (language, generator) across batches, incrementally reconfigured, to eliminate the fixed per-batch nested-configure cost. The main remaining performance lever at module level. - [ ] **Per-entry `DEFINITIONS` + survey partitioning.** A type may need its own flag (`off_t` under `-D_FILE_OFFSET_BITS=64`); since a survey TU has one preprocessor environment, group surveyed queries by flag-set and emit one survey compile per group. - [ ] **Intra-batch dependency/select.** The genuinely hard case: `off_t`'s flag is *conditional on another query computed in the same batch*. Needs either speculative dual-probing with select-at-publish, or an explicit dependency edge. Would appreciate opinions before committing to a design. - [ ] **Implicit batching shim.** Override `check_include_file` &co. to enqueue into an ambient batch that auto-flushes on first read — zero call-site changes. Highest adoption value, trickiest correctness. ### Module coverage map | Module | Disposition | |---|---| | `CheckIncludeFile` / `CheckIncludeFiles` / `CheckIncludeFileCXX` | survey (`INCLUDE`) — implemented | | `CheckSymbolExists` / `CheckCXXSymbolExists` | probe (`SYMBOL`) — implemented; C++-free only if re-expressed as `CALL` | | `CheckFunctionExists` | link probe (`FUNCTION`) — implemented | | `CheckTypeSize` | survey + existence probe — implemented; probe half removable in C++ | | `CheckStructHasMember` | proposed `MEMBER` (free in C++) | | `Check{,C,CXX,Fortran}SourceCompiles` | probe (`COMPILES`) — implemented | | `Check{,C,CXX,Fortran}SourceRuns` / `try_run` | proposed `RUNS` | | `Check{,C,CXX,Fortran}CompilerFlag` | proposed `COMPILE_FLAG` | | `CheckLinkerFlag` | proposed `LINK_FLAG` | | `CheckLibraryExists` | proposed `LIBRARY` | | `CheckVariableExists` | proposed `VARIABLE` | | `CheckPrototypeDefinition` | proposed `PROTOTYPE` | | `TestBigEndian` | surveyable outright (byte-order macro / constant expression) | | `CheckPIESupported`, `CheckIPOSupported` | reducible but one-shot — low priority | | `CheckFortranFunctionExists` | Fortran probe — low priority to me, I do not have any Fortran knowledge to implement this properly yet | | `check_language` | **out of scope** — compiler discovery | ## Open questions for the team 1. **Is a module the right home, or should this eventually be native?** The module-level prototype takes 457 nested try-compiles to 59. The residual is precisely what only a native implementation can remove: a persistent internal probe context instead of per-batch throwaway projects, and a `try_compile_async`-style primitive for the scattered conditional checks that don't fit a declarative batch. Is there appetite for the native direction, or should this stay a module? 2. **`INCLUDE` semantics.** The survey tests header *findability* via `__has_include`; `check_include_file` compiles the inclusion. These agree for self-contained system headers but can differ for headers needing prerequisites. Document as-is, or offer a compile-mode variant? 3. **`SYMBOL` is compile-only** (declaration check, no link), matching `check_symbol_exists` for libc/system symbols. Want a link-checking variant? 4. **A recurring smell worth raising separately:** a number of `try_run` checks in the wild are compile-time-constant properties in disguise (type signedness, size relations, endianness) and disable themselves when cross-compiling for no reason. `HAVE_TIME_T_UNSIGNED` in cmcurl is one — `(time_t)-1 > 0` is a constant expression. Worth auditing independently of this proposal. 5. **On-disk layout.** Probe projects currently live under `CMakeFiles/CheckBatch/`. Preferred convention?