Commit dcf9f4d2 authored by Kyle Edwards's avatar Kyle Edwards Committed by Brad King

Ninja Multi-Config: Add support for cross-config custom commands

Co-Author: Brad King <brad.king@kitware.com>
parent 15467f12
......@@ -398,3 +398,12 @@ after linking.
will run ``someHasher`` after linking ``myPlugin``, e.g. to produce a ``.c``
file containing code to check the hash of ``myPlugin`` that the ``myExe``
executable can use to verify it before loading.
Ninja Multi-Config
^^^^^^^^^^^^^^^^^^
.. versionadded:: 3.20
``add_custom_command`` supports the :generator:`Ninja Multi-Config`
generator's cross-config capabilities. See the generator documentation
for more information.
......@@ -168,3 +168,12 @@ The options are:
.. versionadded:: 3.13
Arguments to ``WORKING_DIRECTORY`` may use
:manual:`generator expressions <cmake-generator-expressions(7)>`.
Ninja Multi-Config
^^^^^^^^^^^^^^^^^^
.. versionadded:: 3.20
``add_custom_target`` supports the :generator:`Ninja Multi-Config`
generator's cross-config capabilities. See the generator documentation
for more information.
......@@ -86,3 +86,47 @@ used to generate ``generated.c``, which would be used to build the ``Debug``
configuration of ``generated``. This is useful for running a release-optimized
version of a generator utility while still building the debug version of the
targets built with the generated code.
Custom Commands
^^^^^^^^^^^^^^^
.. versionadded:: 3.20
The ``Ninja Multi-Config`` generator adds extra capabilities to
:command:`add_custom_command` and :command:`add_custom_target` through its
cross-config mode. The ``COMMAND``, ``DEPENDS``, and ``WORKING_DIRECTORY``
arguments can be evaluated in the context of either the "command config" (the
"native" configuration of the ``build-<Config>.ninja`` file in use) or the
"output config" (the configuration used to evaluate the ``OUTPUT`` and
``BYPRODUCTS``).
If either ``OUTPUT`` or ``BYPRODUCTS`` names a path that is common to
more than one configuration (e.g. it does not use any generator expressions),
all arguments are evaluated in the command config by default.
If all ``OUTPUT`` and ``BYPRODUCTS`` paths are unique to each configuration
(e.g. by using the ``$<CONFIG>`` generator expression), the first argument of
``COMMAND`` is still evaluated in the command config by default, while all
subsequent arguments, as well as the arguments to ``DEPENDS`` and
``WORKING_DIRECTORY``, are evaluated in the output config. These defaults can
be overridden with the ``$<OUTPUT_CONFIG:...>`` and ``$<COMMAND_CONFIG:...>``
generator-expressions. Note that if a target is specified by its name in
``DEPENDS``, or as the first argument of ``COMMAND``, it is always evaluated
in the command config, even if it is wrapped in ``$<OUTPUT_CONFIG:...>``
(because its plain name is not a generator expression).
As an example, consider the following:
.. code-block:: cmake
add_custom_command(
OUTPUT "$<CONFIG>.txt"
COMMAND generator "$<CONFIG>.txt" "$<OUTPUT_CONFIG:$<CONFIG>>" "$<COMMAND_CONFIG:$<CONFIG>>"
DEPENDS tgt1 "$<TARGET_FILE:tgt2>" "$<OUTPUT_CONFIG:$<TARGET_FILE:tgt3>>" "$<COMMAND_CONFIG:$<TARGET_FILE:tgt4>>"
)
Assume that ``generator``, ``tgt1``, ``tgt2``, ``tgt3``, and ``tgt4`` are all
executable targets, and assume that ``$<CONFIG>.txt`` is built in the ``Debug``
output config using the ``Release`` command config. The ``Release`` build of
the ``generator`` target is called with ``Debug.txt Debug Release`` as
arguments. The command depends on the ``Release`` builds of ``tgt1`` and
``tgt4``, and the ``Debug`` builds of ``tgt2`` and ``tgt3``.
......@@ -816,6 +816,24 @@ Output-Related Expressions
``;`` on Windows). Be sure to enclose the argument containing this genex
in double quotes in CMake source code so that ``;`` does not split arguments.
``$<OUTPUT_CONFIG:...>``
.. versionadded:: 3.20
Only valid in :command:`add_custom_command` and :command:`add_custom_target`
as the outer-most generator expression in an argument.
With the :generator:`Ninja Multi-Config` generator, generator expressions
in ``...`` are evaluated using the custom command's "output config".
With other generators, the content of ``...`` is evaluated normally.
``$<COMMAND_CONFIG:...>``
.. versionadded:: 3.20
Only valid in :command:`add_custom_command` and :command:`add_custom_target`
as the outer-most generator expression in an argument.
With the :generator:`Ninja Multi-Config` generator, generator expressions
in ``...`` are evaluated using the custom command's "command config".
With other generators, the content of ``...`` is evaluated normally.
Debugging
=========
......
......@@ -4,3 +4,8 @@ custom-command-output-genex
* :command:`add_custom_command` and :command:`add_custom_target` now
support :manual:`generator expressions <cmake-generator-expressions(7)>`
in their ``OUTPUT`` and ``BYPRODUCTS`` options.
Their ``COMMAND``, ``WORKING_DIRECTORY``, and ``DEPENDS`` options gained
support for new generator expressions ``$<COMMAND_CONFIG:...>`` and
``$<OUTPUT_CONFIG:...>`` that control cross-config handling when using
the :generator:`Ninja Multi-Config` generator.
......@@ -7,7 +7,9 @@
#include <utility>
#include <cm/optional>
#include <cm/string_view>
#include <cmext/algorithm>
#include <cmext/string_view>
#include "cmCryptoHash.h"
#include "cmCustomCommand.h"
......@@ -24,15 +26,95 @@
#include "cmTransformDepfile.h"
namespace {
std::string EvaluateSplitConfigGenex(
cm::string_view input, cmGeneratorExpression const& ge, cmLocalGenerator* lg,
bool useOutputConfig, std::string const& outputConfig,
std::string const& commandConfig,
std::set<BT<std::pair<std::string, bool>>>* utils = nullptr)
{
std::string result;
while (!input.empty()) {
// Copy non-genex content directly to the result.
std::string::size_type pos = input.find("$<");
result += input.substr(0, pos);
if (pos == std::string::npos) {
break;
}
input = input.substr(pos);
// Find the balanced end of this regex.
size_t nestingLevel = 1;
for (pos = 2; pos < input.size(); ++pos) {
cm::string_view cur = input.substr(pos);
if (cmHasLiteralPrefix(cur, "$<")) {
++nestingLevel;
++pos;
continue;
}
if (cmHasLiteralPrefix(cur, ">")) {
--nestingLevel;
if (nestingLevel == 0) {
++pos;
break;
}
}
}
// Split this genex from following input.
cm::string_view genex = input.substr(0, pos);
input = input.substr(pos);
// Convert an outer COMMAND_CONFIG or OUTPUT_CONFIG to the matching config.
std::string const* config =
useOutputConfig ? &outputConfig : &commandConfig;
if (nestingLevel == 0) {
static cm::string_view const COMMAND_CONFIG = "$<COMMAND_CONFIG:"_s;
static cm::string_view const OUTPUT_CONFIG = "$<OUTPUT_CONFIG:"_s;
if (cmHasPrefix(genex, COMMAND_CONFIG)) {
genex.remove_prefix(COMMAND_CONFIG.size());
genex.remove_suffix(1);
useOutputConfig = false;
config = &commandConfig;
} else if (cmHasPrefix(genex, OUTPUT_CONFIG)) {
genex.remove_prefix(OUTPUT_CONFIG.size());
genex.remove_suffix(1);
useOutputConfig = true;
config = &outputConfig;
}
}
// Evaluate this genex in the selected configuration.
std::unique_ptr<cmCompiledGeneratorExpression> cge =
ge.Parse(std::string(genex));
result += cge->Evaluate(lg, *config);
// Record targets referenced by the genex.
if (utils) {
// FIXME: What is the proper condition for a cross-dependency?
bool const cross = !useOutputConfig;
for (cmGeneratorTarget* gt : cge->GetTargets()) {
utils->emplace(BT<std::pair<std::string, bool>>(
{ gt->GetName(), cross }, cge->GetBacktrace()));
}
}
}
return result;
}
std::vector<std::string> EvaluateDepends(std::vector<std::string> const& paths,
cmGeneratorExpression const& ge,
cmLocalGenerator* lg,
std::string const& config)
std::string const& outputConfig,
std::string const& commandConfig)
{
std::vector<std::string> depends;
for (std::string const& p : paths) {
std::unique_ptr<cmCompiledGeneratorExpression> cge = ge.Parse(p);
std::string const& ep = cge->Evaluate(lg, config);
std::string const& ep =
EvaluateSplitConfigGenex(p, ge, lg, /*useOutputConfig=*/true,
/*outputConfig=*/outputConfig,
/*commandConfig=*/commandConfig);
cm::append(depends, cmExpandedList(ep));
}
for (std::string& p : depends) {
......@@ -59,12 +141,12 @@ std::vector<std::string> EvaluateOutputs(std::vector<std::string> const& paths,
}
}
cmCustomCommandGenerator::cmCustomCommandGenerator(cmCustomCommand const& cc,
std::string config,
cmLocalGenerator* lg,
bool transformDepfile)
cmCustomCommandGenerator::cmCustomCommandGenerator(
cmCustomCommand const& cc, std::string config, cmLocalGenerator* lg,
bool transformDepfile, cm::optional<std::string> crossConfig)
: CC(&cc)
, Config(std::move(config))
, OutputConfig(crossConfig ? *crossConfig : config)
, CommandConfig(std::move(config))
, LG(lg)
, OldStyle(cc.GetEscapeOldStyle())
, MakeVars(cc.GetEscapeAllowMakeVars())
......@@ -75,18 +157,20 @@ cmCustomCommandGenerator::cmCustomCommandGenerator(cmCustomCommand const& cc,
const cmCustomCommandLines& cmdlines = this->CC->GetCommandLines();
for (cmCustomCommandLine const& cmdline : cmdlines) {
cmCustomCommandLine argv;
// For the command itself, we default to the COMMAND_CONFIG.
bool useOutputConfig = false;
for (std::string const& clarg : cmdline) {
std::unique_ptr<cmCompiledGeneratorExpression> cge = ge.Parse(clarg);
std::string parsed_arg = cge->Evaluate(this->LG, this->Config);
for (cmGeneratorTarget* gt : cge->GetTargets()) {
this->Utilities.emplace(BT<std::pair<std::string, bool>>(
{ gt->GetName(), true }, cge->GetBacktrace()));
}
std::string parsed_arg = EvaluateSplitConfigGenex(
clarg, ge, this->LG, useOutputConfig, this->OutputConfig,
this->CommandConfig, &this->Utilities);
if (this->CC->GetCommandExpandLists()) {
cm::append(argv, cmExpandedList(parsed_arg));
} else {
argv.push_back(std::move(parsed_arg));
}
// For remaining arguments, we default to the OUTPUT_CONFIG.
useOutputConfig = true;
}
if (!argv.empty()) {
......@@ -94,8 +178,10 @@ cmCustomCommandGenerator::cmCustomCommandGenerator(cmCustomCommand const& cc,
// collect the target to add a target-level dependency on it.
cmGeneratorTarget* gt = this->LG->FindGeneratorTargetToUse(argv.front());
if (gt && gt->GetType() == cmStateEnums::EXECUTABLE) {
// FIXME: What is the proper condition for a cross-dependency?
bool const cross = true;
this->Utilities.emplace(BT<std::pair<std::string, bool>>(
{ gt->GetName(), true }, cc.GetBacktrace()));
{ gt->GetName(), cross }, cc.GetBacktrace()));
}
} else {
// Later code assumes at least one entry exists, but expanding
......@@ -137,16 +223,18 @@ cmCustomCommandGenerator::cmCustomCommandGenerator(cmCustomCommand const& cc,
this->CommandLines.push_back(std::move(argv));
}
this->Outputs = EvaluateOutputs(cc.GetOutputs(), ge, this->LG, this->Config);
this->Outputs =
EvaluateOutputs(cc.GetOutputs(), ge, this->LG, this->OutputConfig);
this->Byproducts =
EvaluateOutputs(cc.GetByproducts(), ge, this->LG, this->Config);
this->Depends = EvaluateDepends(cc.GetDepends(), ge, this->LG, this->Config);
EvaluateOutputs(cc.GetByproducts(), ge, this->LG, this->OutputConfig);
this->Depends = EvaluateDepends(cc.GetDepends(), ge, this->LG,
this->OutputConfig, this->CommandConfig);
const std::string& workingdirectory = this->CC->GetWorkingDirectory();
if (!workingdirectory.empty()) {
std::unique_ptr<cmCompiledGeneratorExpression> cge =
ge.Parse(workingdirectory);
this->WorkingDirectory = cge->Evaluate(this->LG, this->Config);
this->WorkingDirectory =
EvaluateSplitConfigGenex(workingdirectory, ge, this->LG, true,
this->OutputConfig, this->CommandConfig);
// Convert working directory to a full path.
if (!this->WorkingDirectory.empty()) {
std::string const& build_dir = this->LG->GetCurrentBinaryDirectory();
......@@ -203,7 +291,7 @@ const char* cmCustomCommandGenerator::GetArgv0Location(unsigned int c) const
(target->IsImported() ||
target->GetProperty("CROSSCOMPILING_EMULATOR") ||
!this->LG->GetMakefile()->IsOn("CMAKE_CROSSCOMPILING"))) {
return target->GetLocation(this->Config).c_str();
return target->GetLocation(this->CommandConfig).c_str();
}
return nullptr;
}
......
......@@ -9,6 +9,8 @@
#include <utility>
#include <vector>
#include <cm/optional>
#include "cmCustomCommandLines.h"
#include "cmListFileCache.h"
......@@ -18,7 +20,8 @@ class cmLocalGenerator;
class cmCustomCommandGenerator
{
cmCustomCommand const* CC;
std::string Config;
std::string OutputConfig;
std::string CommandConfig;
cmLocalGenerator* LG;
bool OldStyle;
bool MakeVars;
......@@ -36,7 +39,8 @@ class cmCustomCommandGenerator
public:
cmCustomCommandGenerator(cmCustomCommand const& cc, std::string config,
cmLocalGenerator* lg, bool transformDepfile = true);
cmLocalGenerator* lg, bool transformDepfile = true,
cm::optional<std::string> crossConfig = {});
cmCustomCommandGenerator(const cmCustomCommandGenerator&) = delete;
cmCustomCommandGenerator(cmCustomCommandGenerator&&) = default;
cmCustomCommandGenerator& operator=(const cmCustomCommandGenerator&) =
......@@ -55,4 +59,7 @@ public:
bool HasOnlyEmptyCommandLines() const;
std::string GetFullDepfile() const;
std::string GetInternalDepfile() const;
const std::string& GetOutputConfig() const { return this->OutputConfig; }
const std::string& GetCommandConfig() const { return this->CommandConfig; }
};
......@@ -3064,7 +3064,7 @@ bool cmTargetTraceDependencies::IsUtility(std::string const& dep)
} else {
// The original name of the dependency was not a full path. It
// must name a target, so add the target-level dependency.
this->GeneratorTarget->Target->AddUtility(util, false);
this->GeneratorTarget->Target->AddUtility(util, true);
return true;
}
}
......
......@@ -56,6 +56,52 @@ std::string const cmGlobalNinjaGenerator::SHELL_NOOP = "cd .";
std::string const cmGlobalNinjaGenerator::SHELL_NOOP = ":";
#endif
bool operator==(
const cmGlobalNinjaGenerator::ByConfig::TargetDependsClosureKey& lhs,
const cmGlobalNinjaGenerator::ByConfig::TargetDependsClosureKey& rhs)
{
return lhs.Target == rhs.Target && lhs.Config == rhs.Config &&
lhs.GenexOutput == rhs.GenexOutput;
}
bool operator!=(
const cmGlobalNinjaGenerator::ByConfig::TargetDependsClosureKey& lhs,
const cmGlobalNinjaGenerator::ByConfig::TargetDependsClosureKey& rhs)
{
return !(lhs == rhs);
}
bool operator<(
const cmGlobalNinjaGenerator::ByConfig::TargetDependsClosureKey& lhs,
const cmGlobalNinjaGenerator::ByConfig::TargetDependsClosureKey& rhs)
{
return lhs.Target < rhs.Target ||
(lhs.Target == rhs.Target &&
(lhs.Config < rhs.Config ||
(lhs.Config == rhs.Config && lhs.GenexOutput < rhs.GenexOutput)));
}
bool operator>(
const cmGlobalNinjaGenerator::ByConfig::TargetDependsClosureKey& lhs,
const cmGlobalNinjaGenerator::ByConfig::TargetDependsClosureKey& rhs)
{
return rhs < lhs;
}
bool operator<=(
const cmGlobalNinjaGenerator::ByConfig::TargetDependsClosureKey& lhs,
const cmGlobalNinjaGenerator::ByConfig::TargetDependsClosureKey& rhs)
{
return !(lhs > rhs);
}
bool operator>=(
const cmGlobalNinjaGenerator::ByConfig::TargetDependsClosureKey& lhs,
const cmGlobalNinjaGenerator::ByConfig::TargetDependsClosureKey& rhs)
{
return rhs <= lhs;
}
void cmGlobalNinjaGenerator::Indent(std::ostream& os, int count)
{
for (int i = 0; i < count; ++i) {
......@@ -1206,23 +1252,30 @@ void cmGlobalNinjaGenerator::AppendTargetDepends(
void cmGlobalNinjaGenerator::AppendTargetDependsClosure(
cmGeneratorTarget const* target, cmNinjaDeps& outputs,
const std::string& config)
const std::string& config, const std::string& fileConfig, bool genexOutput)
{
cmNinjaOuts outs;
this->AppendTargetDependsClosure(target, outs, config, true);
this->AppendTargetDependsClosure(target, outs, config, fileConfig,
genexOutput, true);
cm::append(outputs, outs);
}
void cmGlobalNinjaGenerator::AppendTargetDependsClosure(
cmGeneratorTarget const* target, cmNinjaOuts& outputs,
const std::string& config, bool omit_self)
const std::string& config, const std::string& fileConfig, bool genexOutput,
bool omit_self)
{
// try to locate the target in the cache
auto find = this->Configs[config].TargetDependsClosures.lower_bound(target);
ByConfig::TargetDependsClosureKey key{
target,
config,
genexOutput,
};
auto find = this->Configs[fileConfig].TargetDependsClosures.lower_bound(key);
if (find == this->Configs[config].TargetDependsClosures.end() ||
find->first != target) {
if (find == this->Configs[fileConfig].TargetDependsClosures.end() ||
find->first != key) {
// We now calculate the closure outputs by inspecting the dependent
// targets recursively.
// For that we have to distinguish between a local result set that is only
......@@ -1232,18 +1285,27 @@ void cmGlobalNinjaGenerator::AppendTargetDependsClosure(
cmNinjaOuts this_outs; // this will be the new cache entry
for (auto const& dep_target : this->GetTargetDirectDepends(target)) {
if (!dep_target->IsInBuildSystem() ||
(target->GetType() != cmStateEnums::UTILITY &&
dep_target->GetType() != cmStateEnums::UTILITY &&
this->EnableCrossConfigBuild() && !dep_target.IsCross())) {
if (!dep_target->IsInBuildSystem()) {
continue;
}
if (!this->IsSingleConfigUtility(target) &&
!this->IsSingleConfigUtility(dep_target) &&
this->EnableCrossConfigBuild() && !dep_target.IsCross() &&
!genexOutput) {
continue;
}
// Collect the dependent targets for _this_ target
this->AppendTargetDependsClosure(dep_target, this_outs, config, false);
if (dep_target.IsCross()) {
this->AppendTargetDependsClosure(dep_target, this_outs, fileConfig,
fileConfig, genexOutput, false);
} else {
this->AppendTargetDependsClosure(dep_target, this_outs, config,
fileConfig, genexOutput, false);
}
}
find = this->Configs[config].TargetDependsClosures.emplace_hint(
find, target, std::move(this_outs));
find = this->Configs[fileConfig].TargetDependsClosures.emplace_hint(
find, key, std::move(this_outs));
}
// now fill the outputs of the final result from the newly generated cache
......@@ -2490,6 +2552,13 @@ std::set<std::string> cmGlobalNinjaGenerator::GetCrossConfigs(
return result;
}
bool cmGlobalNinjaGenerator::IsSingleConfigUtility(
cmGeneratorTarget const* target) const
{
return target->GetType() == cmStateEnums::UTILITY &&
!this->PerConfigUtilityTargets.count(target->GetName());
}
const char* cmGlobalNinjaMultiGenerator::NINJA_COMMON_FILE =
"CMakeFiles/common.ninja";
const char* cmGlobalNinjaMultiGenerator::NINJA_FILE_EXTENSION = ".ninja";
......
......@@ -330,10 +330,14 @@ public:
cmNinjaTargetDepends depends);
void AppendTargetDependsClosure(cmGeneratorTarget const* target,
cmNinjaDeps& outputs,
const std::string& config);
const std::string& config,
const std::string& fileConfig,
bool genexOutput);
void AppendTargetDependsClosure(cmGeneratorTarget const* target,
cmNinjaOuts& outputs,
const std::string& config, bool omit_self);
const std::string& config,
const std::string& fileConfig,
bool genexOutput, bool omit_self);
void AppendDirectoryForConfig(const std::string& prefix,
const std::string& config,
......@@ -430,6 +434,18 @@ public:
return this->DefaultConfigs;
}
const std::set<std::string>& GetPerConfigUtilityTargets() const
{
return this->PerConfigUtilityTargets;
}
void AddPerConfigUtilityTarget(const std::string& name)
{
this->PerConfigUtilityTargets.insert(name);
}
bool IsSingleConfigUtility(cmGeneratorTarget const* target) const;
protected:
void Generate() override;
......@@ -523,6 +539,9 @@ private:
/// The mapping from source file to assumed dependencies.
std::map<std::string, std::set<std::string>> AssumedSourceDependencies;
/// Utility targets which have per-config outputs
std::set<std::string> PerConfigUtilityTargets;
struct TargetAlias
{
cmGeneratorTarget* GeneratorTarget;
......@@ -563,7 +582,14 @@ private:
/// The set of custom commands we have seen.
std::set<cmCustomCommand const*> CustomCommands;
std::map<cmGeneratorTarget const*, cmNinjaOuts> TargetDependsClosures;
struct TargetDependsClosureKey
{
cmGeneratorTarget const* Target;
std::string Config;
bool GenexOutput;
};
std::map<TargetDependsClosureKey, cmNinjaOuts> TargetDependsClosures;
TargetAliasMap TargetAliases;
......@@ -572,6 +598,19 @@ private:
std::map<std::string, ByConfig> Configs;
cmNinjaDeps ByproductsForCleanTarget;
friend bool operator==(const ByConfig::TargetDependsClosureKey& lhs,
const ByConfig::TargetDependsClosureKey& rhs);
friend bool operator!=(const ByConfig::TargetDependsClosureKey& lhs,
const ByConfig::TargetDependsClosureKey& rhs);
friend bool operator<(const ByConfig::TargetDependsClosureKey& lhs,
const ByConfig::TargetDependsClosureKey& rhs);
friend bool operator>(const ByConfig::TargetDependsClosureKey& lhs,
const ByConfig::TargetDependsClosureKey& rhs);
friend bool operator<=(const ByConfig::TargetDependsClosureKey& lhs,
const ByConfig::TargetDependsClosureKey& rhs);
friend bool operator>=(const ByConfig::TargetDependsClosureKey& lhs,
const ByConfig::TargetDependsClosureKey& rhs);
};
class cmGlobalNinjaMultiGenerator : public cmGlobalNinjaGenerator
......
......@@ -4128,7 +4128,8 @@ void AddUtilityCommand(cmLocalGenerator& lg, const cmListFileBacktrace& lfbt,
}
// Create the generated symbolic output name of the utility target.
std::string output = lg.CreateUtilityOutput(target->GetName());
std::string output =
lg.CreateUtilityOutput(target->GetName(), byproducts, lfbt);
std::string no_main_dependency;
cmImplicitDependsList no_implicit_depends;
......@@ -4235,7 +4236,8 @@ cmSourceFile* cmLocalGenerator::GetSourceFileWithOutput(
}
std::string cmLocalGenerator::CreateUtilityOutput(
std::string const& targetName)
std::string const& targetName, std::vector<std::string> const&,
cmListFileBacktrace const&)
{
std::string force =
cmStrCat(this->GetCurrentBinaryDirectory(), "/CMakeFiles/", targetName);
......
......@@ -364,7 +364,9 @@ public:
bool command_expand_lists = false, const std::string& job_pool = "",
bool stdPipesUTF8 = false);
std::string CreateUtilityOutput(std::string const& targetName);