Commit fedfe763 authored by Brad King's avatar Brad King Committed by Kitware Robot
Browse files

Merge topic 'custom-command-output-genex'

c257c254 add_custom_{command,target}: Add genex support to OUTPUT and BYPRODUCTS
f36af922 cmLocalGenerator: Evaluate generator expressions in custom command outputs
c887cefd cmLocalGenerator: Simplify custom command output cmSourceFile creation
947ba01b cmLocalGenerator: Factor out helper to expand custom command output paths
1902d28e cmLocalGenerator: Refactor UpdateOutputToSourceMap to avoid boolean trap
e4034eab cmLocalGenerator: Re-order logic in CreateGeneratedSource
706c4830 cmCustomCommandGenerator: Treat relative outputs w.r.t. build dir
5d23c544

 cmCustomCommandGenerator: Refactor OUTPUT and DEPENDS path evaluation
...
Acked-by: Kitware Robot's avatarKitware Robot <kwrobot@kitware.com>
Acked-by: Kyle Edwards's avatarKyle Edwards <kyle.edwards@kitware.com>
Acked-by: Pavel Solodovnikov's avatarPavel Solodovnikov <hellyeahdominate@gmail.com>
Acked-by: Ben Boeckel's avatarBen Boeckel <ben.boeckel@kitware.com>
Merge-request: !5402
parents 979af92e c257c254
Pipeline #204785 waiting for manual action with stages
in 5 minutes and 50 seconds
......@@ -46,6 +46,12 @@ The options are:
Append the ``COMMAND`` and ``DEPENDS`` option values to the custom
command for the first output specified. There must have already
been a previous call to this command with the same output.
If the previous call specified the output via a generator expression,
the output specified by the current call must match in at least one
configuration after evaluating generator expressions. In this case,
the appended commands and dependencies apply to all configurations.
The ``COMMENT``, ``MAIN_DEPENDENCY``, and ``WORKING_DIRECTORY``
options are currently ignored when APPEND is given, but may be
used in the future.
......@@ -73,6 +79,9 @@ The options are:
The :ref:`Makefile Generators` will remove ``BYPRODUCTS`` and other
:prop_sf:`GENERATED` files during ``make clean``.
Since CMake 3.20, arguments to ``BYPRODUCTS`` may use
:manual:`generator expressions <cmake-generator-expressions(7)>`.
``COMMAND``
Specify the command-line(s) to execute at build time.
If more than one ``COMMAND`` is specified they will be executed in order,
......@@ -220,6 +229,9 @@ The options are:
as a file on disk it should be marked with the :prop_sf:`SYMBOLIC`
source file property.
Since CMake 3.20, arguments to ``OUTPUT`` may use
:manual:`generator expressions <cmake-generator-expressions(7)>`.
``USES_TERMINAL``
.. versionadded:: 3.2
......@@ -259,6 +271,44 @@ The options are:
``DEPFILE`` should also be relative to :variable:`CMAKE_CURRENT_BINARY_DIR`
(see policy :policy:`CMP0116`.)
Examples: Generating Files
^^^^^^^^^^^^^^^^^^^^^^^^^^
Custom commands may be used to generate source files.
For example, the code:
.. code-block:: cmake
add_custom_command(
OUTPUT out.c
COMMAND someTool -i ${CMAKE_CURRENT_SOURCE_DIR}/in.txt
-o out.c
DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/in.txt
VERBATIM)
add_library(myLib out.c)
adds a custom command to run ``someTool`` to generate ``out.c`` and then
compile the generated source as part of a library. The generation rule
will re-run whenever ``in.txt`` changes.
Since CMake 3.20, one may use generator expressions to specify
per-configuration outputs. For example, the code:
.. code-block:: cmake
add_custom_command(
OUTPUT "out-$<CONFIG>.c"
COMMAND someTool -i ${CMAKE_CURRENT_SOURCE_DIR}/in.txt
-o "out-$<CONFIG>.c"
-c "$<CONFIG>"
DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/in.txt
VERBATIM)
add_library(myLib "out-$<CONFIG>.c")
adds a custom command to run ``someTool`` to generate ``out-<config>.c``,
where ``<config>`` is the build configuration, and then compile the generated
source as part of a library.
Build Events
^^^^^^^^^^^^
......@@ -308,3 +358,39 @@ of the following is specified:
configuration and no "empty-string-command" will be added.
This allows to add individual build events for every configuration.
Examples: Build Events
^^^^^^^^^^^^^^^^^^^^^^
A ``POST_BUILD`` event may be used to post-process a binary after linking.
For example, the code:
.. code-block:: cmake
add_executable(myExe myExe.c)
add_custom_command(
TARGET myExe POST_BUILD
COMMAND someHasher -i "$<TARGET_FILE:myExe>"
-o "$<TARGET_FILE:myExe>.hash"
VERBATIM)
will run ``someHasher`` to produce a ``.hash`` file next to the executable
after linking.
Since CMake 3.20, one may use generator expressions to specify
per-configuration byproducts. For example, the code:
.. code-block:: cmake
add_library(myPlugin MODULE myPlugin.c)
add_custom_command(
TARGET myPlugin POST_BUILD
COMMAND someHasher -i "$<TARGET_FILE:myPlugin>"
--as-code "myPlugin-hash-$<CONFIG>.c"
BYPRODUCTS "myPlugin-hash-$<CONFIG>.c"
VERBATIM)
add_executable(myExe myExe.c "myPlugin-hash-$<CONFIG>.c")
will run ``someHasher`` after linking ``myPlugin``, e.g. to produce a ``.c``
file containing code to check the hash of ``myPlugin`` that the ``myExe``
executable can use to verify it before loading.
......@@ -54,6 +54,9 @@ The options are:
The :ref:`Makefile Generators` will remove ``BYPRODUCTS`` and other
:prop_sf:`GENERATED` files during ``make clean``.
Since CMake 3.20, arguments to ``BYPRODUCTS`` may use
:manual:`generator expressions <cmake-generator-expressions(7)>`.
``COMMAND``
Specify the command-line(s) to execute at build time.
If more than one ``COMMAND`` is specified they will be executed in order,
......
custom-command-output-genex
---------------------------
* :command:`add_custom_command` and :command:`add_custom_target` now
support :manual:`generator expressions <cmake-generator-expressions(7)>`
in their ``OUTPUT`` and ``BYPRODUCTS`` options.
......@@ -181,8 +181,6 @@ set(SRCS
cmBinUtilsWindowsPEObjdumpGetRuntimeDependenciesTool.h
cmCacheManager.cxx
cmCacheManager.h
cmCheckCustomOutputs.h
cmCheckCustomOutputs.cxx
cmCLocaleEnvironmentScope.h
cmCLocaleEnvironmentScope.cxx
cmCMakePath.h
......
......@@ -5,11 +5,11 @@
#include <sstream>
#include <unordered_set>
#include "cmCheckCustomOutputs.h"
#include "cmCustomCommand.h"
#include "cmCustomCommandLines.h"
#include "cmCustomCommandTypes.h"
#include "cmExecutionStatus.h"
#include "cmGeneratorExpression.h"
#include "cmGlobalGenerator.h"
#include "cmMakefile.h"
#include "cmMessageType.h"
......@@ -188,7 +188,8 @@ bool cmAddCustomCommandCommand(std::vector<std::string> const& args,
case doing_output:
case doing_outputs:
case doing_byproducts:
if (!cmSystemTools::FileIsFullPath(copy)) {
if (!cmSystemTools::FileIsFullPath(copy) &&
cmGeneratorExpression::Find(copy) != 0) {
// This is an output to be generated, so it should be
// under the build tree.
filename = cmStrCat(mf.GetCurrentBinaryDirectory(), '/');
......@@ -296,13 +297,6 @@ bool cmAddCustomCommandCommand(std::vector<std::string> const& args,
return false;
}
// Make sure the output names and locations are safe.
if (!cmCheckCustomOutputs(output, "OUTPUT", status) ||
!cmCheckCustomOutputs(outputs, "OUTPUTS", status) ||
!cmCheckCustomOutputs(byproducts, "BYPRODUCTS", status)) {
return false;
}
// Check for an append request.
if (append) {
mf.AppendCustomCommandToOutput(output[0], depends, implicit_depends,
......
......@@ -4,7 +4,6 @@
#include <utility>
#include "cmCheckCustomOutputs.h"
#include "cmCustomCommandLines.h"
#include "cmExecutionStatus.h"
#include "cmGeneratorExpression.h"
......@@ -120,12 +119,16 @@ bool cmAddCustomTargetCommand(std::vector<std::string> const& args,
break;
case doing_byproducts: {
std::string filename;
if (!cmSystemTools::FileIsFullPath(copy)) {
if (!cmSystemTools::FileIsFullPath(copy) &&
cmGeneratorExpression::Find(copy) != 0) {
filename = cmStrCat(mf.GetCurrentBinaryDirectory(), '/');
}
filename += copy;
cmSystemTools::ConvertToUnixSlashes(filename);
byproducts.push_back(cmSystemTools::CollapseFullPath(filename));
if (cmSystemTools::FileIsFullPath(filename)) {
filename = cmSystemTools::CollapseFullPath(filename);
}
byproducts.push_back(filename);
} break;
case doing_depends: {
std::string dep = copy;
......@@ -206,11 +209,6 @@ bool cmAddCustomTargetCommand(std::vector<std::string> const& args,
return false;
}
// Make sure the byproduct names and locations are safe.
if (!cmCheckCustomOutputs(byproducts, "BYPRODUCTS", status)) {
return false;
}
// Add the utility target to the makefile.
bool escapeOldStyle = !verbatim;
cmTarget* target = mf.AddUtilityCommand(
......
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file Copyright.txt or https://cmake.org/licensing for details. */
#include "cmCheckCustomOutputs.h"
#include "cmExecutionStatus.h"
#include "cmMakefile.h"
#include "cmStringAlgorithms.h"
#include "cmSystemTools.h"
bool cmCheckCustomOutputs(const std::vector<std::string>& outputs,
cm::string_view keyword, cmExecutionStatus& status)
{
cmMakefile& mf = status.GetMakefile();
for (std::string const& o : outputs) {
// Make sure the file will not be generated into the source
// directory during an out of source build.
if (!mf.CanIWriteThisFile(o)) {
status.SetError(
cmStrCat("attempted to have a file\n ", o,
"\nin a source directory as an output of custom command."));
cmSystemTools::SetFatalErrorOccured();
return false;
}
// Make sure the output file name has no invalid characters.
std::string::size_type pos = o.find_first_of("#<>");
if (pos != std::string::npos) {
status.SetError(cmStrCat("called with ", keyword, " containing a \"",
o[pos], "\". This character is not allowed."));
return false;
}
}
return true;
}
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file Copyright.txt or https://cmake.org/licensing for details. */
#pragma once
#include "cmConfigure.h" // IWYU pragma: keep
#include <string>
#include <vector>
#include <cm/string_view>
class cmExecutionStatus;
bool cmCheckCustomOutputs(const std::vector<std::string>& outputs,
cm::string_view keyword, cmExecutionStatus& status);
......@@ -24,22 +24,38 @@
#include "cmTransformDepfile.h"
namespace {
void AppendPaths(const std::vector<std::string>& inputs,
cmGeneratorExpression const& ge, cmLocalGenerator* lg,
std::string const& config, std::vector<std::string>& output)
std::vector<std::string> EvaluateDepends(std::vector<std::string> const& paths,
cmGeneratorExpression const& ge,
cmLocalGenerator* lg,
std::string const& config)
{
for (std::string const& in : inputs) {
std::unique_ptr<cmCompiledGeneratorExpression> cge = ge.Parse(in);
std::vector<std::string> result =
cmExpandedList(cge->Evaluate(lg, config));
for (std::string& it : result) {
cmSystemTools::ConvertToUnixSlashes(it);
if (cmSystemTools::FileIsFullPath(it)) {
it = cmSystemTools::CollapseFullPath(it);
}
std::vector<std::string> depends;
for (std::string const& p : paths) {
std::unique_ptr<cmCompiledGeneratorExpression> cge = ge.Parse(p);
std::string const& ep = cge->Evaluate(lg, config);
cm::append(depends, cmExpandedList(ep));
}
for (std::string& p : depends) {
if (cmSystemTools::FileIsFullPath(p)) {
p = cmSystemTools::CollapseFullPath(p);
} else {
cmSystemTools::ConvertToUnixSlashes(p);
}
cm::append(output, result);
}
return depends;
}
std::vector<std::string> EvaluateOutputs(std::vector<std::string> const& paths,
cmGeneratorExpression const& ge,
cmLocalGenerator* lg,
std::string const& config)
{
std::vector<std::string> outputs;
for (std::string const& p : paths) {
std::unique_ptr<cmCompiledGeneratorExpression> cge = ge.Parse(p);
cm::append(outputs, lg->ExpandCustomCommandOutputPaths(*cge, config));
}
return outputs;
}
}
......@@ -121,9 +137,10 @@ cmCustomCommandGenerator::cmCustomCommandGenerator(cmCustomCommand const& cc,
this->CommandLines.push_back(std::move(argv));
}
AppendPaths(cc.GetByproducts(), ge, this->LG, this->Config,
this->Byproducts);
AppendPaths(cc.GetDepends(), ge, this->LG, this->Config, this->Depends);
this->Outputs = EvaluateOutputs(cc.GetOutputs(), ge, this->LG, this->Config);
this->Byproducts =
EvaluateOutputs(cc.GetByproducts(), ge, this->LG, this->Config);
this->Depends = EvaluateDepends(cc.GetDepends(), ge, this->LG, this->Config);
const std::string& workingdirectory = this->CC->GetWorkingDirectory();
if (!workingdirectory.empty()) {
......@@ -326,7 +343,7 @@ std::string cmCustomCommandGenerator::GetWorkingDirectory() const
std::vector<std::string> const& cmCustomCommandGenerator::GetOutputs() const
{
return this->CC->GetOutputs();
return this->Outputs;
}
std::vector<std::string> const& cmCustomCommandGenerator::GetByproducts() const
......
......@@ -24,6 +24,7 @@ class cmCustomCommandGenerator
bool MakeVars;
cmCustomCommandLines CommandLines;
std::vector<std::vector<std::string>> EmulatorsWithArguments;
std::vector<std::string> Outputs;
std::vector<std::string> Byproducts;
std::vector<std::string> Depends;
std::string WorkingDirectory;
......
......@@ -17,9 +17,11 @@
#include <cm/memory>
#include <cm/string_view>
#include <cmext/algorithm>
#include <cmext/string_view>
#include "cmsys/RegularExpression.hxx"
#include "cmAlgorithms.h"
#include "cmComputeLinkInformation.h"
#include "cmCustomCommand.h"
#include "cmCustomCommandGenerator.h"
......@@ -3812,38 +3814,95 @@ void cmLocalGenerator::GenerateFrameworkInfoPList(
}
namespace {
cm::string_view CustomOutputRoleKeyword(cmLocalGenerator::OutputRole role)
{
return (role == cmLocalGenerator::OutputRole::Primary ? "OUTPUT"_s
: "BYPRODUCTS"_s);
}
void CreateGeneratedSource(cmLocalGenerator& lg, const std::string& output,
cmLocalGenerator::OutputRole role,
cmCommandOrigin origin,
const cmListFileBacktrace& lfbt)
{
if (cmGeneratorExpression::Find(output) == std::string::npos) {
// Outputs without generator expressions from the project are already
// created and marked as generated. Do not mark them again, because
// other commands might have overwritten the property.
if (origin == cmCommandOrigin::Generator) {
lg.GetMakefile()->GetOrCreateGeneratedSource(output);
}
} else {
if (cmGeneratorExpression::Find(output) != std::string::npos) {
lg.GetCMakeInstance()->IssueMessage(
MessageType::FATAL_ERROR,
"Generator expressions in custom command outputs are not implemented!",
lfbt);
return;
}
// Make sure the file will not be generated into the source
// directory during an out of source build.
if (!lg.GetMakefile()->CanIWriteThisFile(output)) {
lg.GetCMakeInstance()->IssueMessage(
MessageType::FATAL_ERROR,
cmStrCat(CustomOutputRoleKeyword(role), " path\n ", output,
"\nin a source directory as an output of custom command."),
lfbt);
return;
}
// Make sure the output file name has no invalid characters.
std::string::size_type pos = output.find_first_of("#<>");
if (pos != std::string::npos) {
lg.GetCMakeInstance()->IssueMessage(
MessageType::FATAL_ERROR,
cmStrCat(CustomOutputRoleKeyword(role), " containing a \"", output[pos],
"\" is not allowed."),
lfbt);
return;
}
// Outputs without generator expressions from the project are already
// created and marked as generated. Do not mark them again, because
// other commands might have overwritten the property.
if (origin == cmCommandOrigin::Generator) {
lg.GetMakefile()->GetOrCreateGeneratedSource(output);
}
}
void CreateGeneratedSources(cmLocalGenerator& lg,
const std::vector<std::string>& outputs,
cmCommandOrigin origin,
const cmListFileBacktrace& lfbt)
std::string ComputeCustomCommandRuleFileName(cmLocalGenerator& lg,
cmListFileBacktrace const& bt,
std::string const& output)
{
for (std::string const& o : outputs) {
CreateGeneratedSource(lg, o, origin, lfbt);
// If the output path has no generator expressions, use it directly.
if (cmGeneratorExpression::Find(output) == std::string::npos) {
return output;
}
// The output path contains a generator expression, but we must choose
// a single source file path to which to attach the custom command.
// Use some heuristics to provie a nice-looking name when possible.
// If the only genex is $<CONFIG>, replace that gracefully.
{
std::string simple = output;
cmSystemTools::ReplaceString(simple, "$<CONFIG>", "(CONFIG)");
if (cmGeneratorExpression::Find(simple) == std::string::npos) {
return simple;
}
}
// If the genex evaluates to the same value in all configurations, use that.
{
std::vector<std::string> allConfigOutputs =
lg.ExpandCustomCommandOutputGenex(output, bt);
if (allConfigOutputs.size() == 1) {
return allConfigOutputs.front();
}
}
// Fall back to a deterministic unique name.
cmCryptoHash h(cmCryptoHash::AlgoSHA256);
return cmStrCat(lg.GetCurrentBinaryDirectory(), "/CMakeFiles/",
h.HashString(output).substr(0, 16));
}
cmSourceFile* AddCustomCommand(
cmLocalGenerator& lg, const cmListFileBacktrace& lfbt,
const std::vector<std::string>& outputs,
cmCommandOrigin origin, const std::vector<std::string>& outputs,
const std::vector<std::string>& byproducts,
const std::vector<std::string>& depends, const std::string& main_dependency,
const cmImplicitDependsList& implicit_depends,
......@@ -3880,7 +3939,8 @@ cmSourceFile* AddCustomCommand(
cmGlobalGenerator* gg = lg.GetGlobalGenerator();
// Construct a rule file associated with the first output produced.
std::string outName = gg->GenerateRuleFile(outputs[0]);
std::string outName = gg->GenerateRuleFile(
ComputeCustomCommandRuleFileName(lg, lfbt, outputs[0]));
// Check if the rule file already exists.
file = mf->GetSource(outName, cmSourceFileLocationKind::Known);
......@@ -3923,7 +3983,10 @@ cmSourceFile* AddCustomCommand(
cc->SetJobPool(job_pool);
file->SetCustomCommand(std::move(cc));
lg.AddSourceOutputs(file, outputs, byproducts);
lg.AddSourceOutputs(file, outputs, cmLocalGenerator::OutputRole::Primary,
lfbt, origin);
lg.AddSourceOutputs(file, byproducts,
cmLocalGenerator::OutputRole::Byproduct, lfbt, origin);
}
return file;
}
......@@ -3967,9 +4030,6 @@ void AddCustomCommandToTarget(cmLocalGenerator& lg,
const std::string& job_pool,
bool command_expand_lists, bool stdPipesUTF8)
{
// Always create the byproduct sources and mark them generated.
CreateGeneratedSources(lg, byproducts, origin, lfbt);
// Add the command to the appropriate build step for the target.
std::vector<std::string> no_output;
cmCustomCommand cc(no_output, byproducts, depends, commandLines, lfbt,
......@@ -3992,7 +4052,7 @@ void AddCustomCommandToTarget(cmLocalGenerator& lg,
break;
}
lg.AddTargetByproducts(target, byproducts);
lg.AddTargetByproducts(target, byproducts, lfbt, origin);
}
cmSourceFile* AddCustomCommandToOutput(
......@@ -4006,14 +4066,11 @@ cmSourceFile* AddCustomCommandToOutput(
bool uses_terminal, bool command_expand_lists, const std::string& depfile,
const std::string& job_pool, bool stdPipesUTF8)
{
// Always create the output sources and mark them generated.
CreateGeneratedSources(lg, outputs, origin, lfbt);
CreateGeneratedSources(lg, byproducts, origin, lfbt);
return AddCustomCommand(
lg, lfbt, outputs, byproducts, depends, main_dependency, implicit_depends,
commandLines, comment, workingDir, replace, escapeOldStyle, uses_terminal,
command_expand_lists, depfile, job_pool, stdPipesUTF8);
return AddCustomCommand(lg, lfbt, origin, outputs, byproducts, depends,
main_dependency, implicit_depends, commandLines,
comment, workingDir, replace, escapeOldStyle,
uses_terminal, command_expand_lists, depfile,
job_pool, stdPipesUTF8);
}
void AppendCustomCommandToOutput(cmLocalGenerator& lg,
......@@ -4024,7 +4081,22 @@ void AppendCustomCommandToOutput(cmLocalGenerator& lg,
const cmCustomCommandLines& commandLines)
{
// Lookup an existing command.
if (cmSourceFile* sf = lg.GetSourceFileWithOutput(output)) {
cmSourceFile* sf = nullptr;
if (cmGeneratorExpression::Find(output) == std::string::npos) {
sf = lg.GetSourceFileWithOutput(output);
} else {
// This output path has a generator expression. Evaluate it to
// find the output for any configurations.
for (std::string const& out :
lg.ExpandCustomCommandOutputGenex(output, lfbt)) {
sf = lg.GetSourceFileWithOutput(out);
if (sf) {
break;
}
}
}
if (sf) {
if (cmCustomCommand* cc = sf->GetCustomCommand()) {
cc->AppendCommands(commandLines);
cc->AppendDepends(depends);
......@@ -4051,10 +4123,6 @@ void AddUtilityCommand(cmLocalGenerator& lg, const cmListFileBacktrace& lfbt,
bool uses_terminal, bool command_expand_lists,
const std::string& job_pool, bool stdPipesUTF8)
{
// Always create the byproduct sources and mark them generated.
CreateGeneratedSource(lg, force.Name, origin, lfbt);
CreateGeneratedSources(lg, byproducts, origin, lfbt);
// Use an empty comment to avoid generation of default comment.
if (!comment) {
comment = "";
......@@ -4063,12 +4131,12 @@ void AddUtilityCommand(cmLocalGenerator& lg, const cmListFileBacktrace& lfbt,
std::string no_main_dependency;
cmImplicitDependsList no_implicit_depends;
cmSourceFile* rule = AddCustomCommand(
lg, lfbt, { force.Name }, byproducts, depends, no_main_dependency,
lg, lfbt, origin, { force.Name }, byproducts, depends, no_main_dependency,
no_implicit_depends, commandLines, comment, workingDir,
/*replace=*/false, escapeOldStyle, uses_terminal, command_expand_lists,
/*depfile=*/"", job_pool, stdPipesUTF8);
if (rule) {
lg.AddTargetByproducts(target, byproducts);
lg.AddTargetByproducts(target, byproducts, lfbt, origin);
}
if (!force.NameCMP0049.empty()) {
......@@ -4166,34 +4234,87 @@ cmSourceFile* cmLocalGenerator::GetSourceFileWithOutput(
return nullptr;
}
std::vector<std::string> cmLocalGenerator::ExpandCustomCommandOutputPaths(
cmCompiledGeneratorExpression const& cge, std::string const& config)
{
std::vector<std::string> paths = cmExpandedList(cge.Evaluate(this, config));
for (std::string& p : paths) {
p = cmSystemTools::CollapseFullPath(p, this->GetCurrentBinaryDirectory());
}
return paths;
}