Commit a7c1e15c authored by Craig Scott's avatar Craig Scott Committed by Brad King

CTest: Rename hardware -> resources for CMake variables, command options

Only changes the user-visible effects of renaming hardware
allocation to resource allocation. Code changes are the minimum
needed to achieve that.
parent 73a40b19
......@@ -17,7 +17,7 @@ Perform the :ref:`CTest Test Step` as a :ref:`Dashboard Client`.
[EXCLUDE_FIXTURE_SETUP <regex>]
[EXCLUDE_FIXTURE_CLEANUP <regex>]
[PARALLEL_LEVEL <level>]
[HARDWARE_SPEC_FILE <file>]
[RESOURCE_SPEC_FILE <file>]
[TEST_LOAD <threshold>]
[SCHEDULE_RANDOM <ON|OFF>]
[STOP_TIME <time-of-day>]
......@@ -83,10 +83,10 @@ The options are:
Specify a positive number representing the number of tests to
be run in parallel.
``HARDWARE_SPEC_FILE <file>``
``RESOURCE_SPEC_FILE <file>``
Specify a
:ref:`hardware specification file <ctest-hardware-specification-file>`. See
:ref:`ctest-hardware-allocation` for more information.
:ref:`resource specification file <ctest-resource-specification-file>`. See
:ref:`ctest-resource-allocation` for more information.
``TEST_LOAD <threshold>``
While running tests in parallel, try not to start tests when they
......
......@@ -90,14 +90,14 @@ Options
See `Label and Subproject Summary`_.
``--hardware-spec-file <file>``
Run CTest with :ref:`hardware allocation <ctest-hardware-allocation>` enabled,
``--resource-spec-file <file>``
Run CTest with :ref:`resource allocation <ctest-resource-allocation>` enabled,
using the
:ref:`hardware specification file <ctest-hardware-specification-file>`
:ref:`resource specification file <ctest-resource-specification-file>`
specified in ``<file>``.
When ``ctest`` is run as a `Dashboard Client`_ this sets the
``HardwareSpecFile`` option of the `CTest Test Step`_.
``ResourceSpecFile`` option of the `CTest Test Step`_.
``--test-load <level>``
While running tests in parallel (e.g. with ``-j``), try not to start
......@@ -967,10 +967,10 @@ Arguments to the command may specify some of the step settings.
Configuration settings include:
``HardwareSpecFile``
``ResourceSpecFile``
Specify a
:ref:`hardware specification file <ctest-hardware-specification-file>`. See
:ref:`ctest-hardware-allocation` for more information.
:ref:`resource specification file <ctest-resource-specification-file>`. See
:ref:`ctest-resource-allocation` for more information.
``LabelsForSubprojects``
Specify a semicolon-separated list of labels that will be treated as
......@@ -1281,22 +1281,22 @@ model is defined as follows:
Test properties.
Can contain keys for each of the supported test properties.
.. _`ctest-hardware-allocation`:
.. _`ctest-resource-allocation`:
Hardware Allocation
Resource Allocation
===================
CTest provides a mechanism for tests to specify the hardware that they need and
how much of it they need, and for users to specify the hardware availiable on
CTest provides a mechanism for tests to specify the resources that they need
in a fine-grained way, and for users to specify the resources availiable on
the running machine. This allows CTest to internally keep track of which
hardware is in use and which is free, scheduling tests in a way that prevents
them from trying to claim hardware that is not available.
resources are in use and which are free, scheduling tests in a way that
prevents them from trying to claim resources that are not available.
A common use case for this feature is for tests that require the use of a GPU.
Multiple tests can simultaneously allocate memory from a GPU, but if too many
tests try to do this at once, some of them will fail to allocate, resulting in
a failed test, even though the test would have succeeded if it had the memory
it needed. By using the hardware allocation feature, each test can specify how
it needed. By using the resource allocation feature, each test can specify how
much memory it requires from a GPU, allowing CTest to schedule tests in a way
that running several of these tests at once does not exhaust the GPU's memory
pool.
......@@ -1312,36 +1312,35 @@ When a test is executed, and slots from a resource are allocated to that test,
tests may assume that they have exclusive use of those slots for the duration
of the test's process.
The CTest hardware allocation feature consists of two inputs:
The CTest resource allocation feature consists of two inputs:
* The :ref:`hardware specification file <ctest-hardware-specification-file>`,
described below, which describes the hardware resources available on the
system, and
* The :ref:`resource specification file <ctest-resource-specification-file>`,
described below, which describes the resources available on the system.
* The :prop_test:`RESOURCE_GROUPS` property of tests, which describes the
resources required by the test
resources required by the test.
When CTest runs a test, the hardware allocated to that test is passed in the
When CTest runs a test, the resources allocated to that test are passed in the
form of a set of
:ref:`environment variables <ctest-hardware-environment-variables>` as
:ref:`environment variables <ctest-resource-environment-variables>` as
described below. Using this information to decide which resource to connect to
is left to the test writer.
The ``RESOURCE_GROUPS`` property tells CTest what resources a test expects
to use grouped in a way meaningful to the test. The test itself must read
the :ref:`environment variables <ctest-hardware-environment-variables>` to
the :ref:`environment variables <ctest-resource-environment-variables>` to
determine which resources have been allocated to each group. For example,
each group may correspond to a process the test will spawn when executed.
.. _`ctest-hardware-specification-file`:
.. _`ctest-resource-specification-file`:
Hardware Specification File
Resource Specification File
---------------------------
The hardware specification file is a JSON file which is passed to CTest, either
on the :manual:`ctest(1)` command line as ``--hardware-spec-file``, or as the
``HARDWARE_SPEC_FILE`` argument of :command:`ctest_test`. The hardware
The resource specification file is a JSON file which is passed to CTest, either
on the :manual:`ctest(1)` command line as ``--resource-spec-file``, or as the
``RESOURCE_SPEC_FILE`` argument of :command:`ctest_test`. The resource
specification file must be a JSON object. All examples in this document assume
the following hardware specification file:
the following resource specification file:
.. code-block:: json
......@@ -1378,11 +1377,11 @@ the following hardware specification file:
The members are:
``local``
A JSON array consisting of CPU sockets present on the system. Currently, only
one socket is supported.
A JSON array of resource sets present on the system. Currently, this array
is restricted to being of size 1.
Each socket is a JSON object with members whose names are equal to the
desired resource types, such as ``gpu``. These names must start with a
Each array element is a JSON object with members whose names are equal to the
desired resource types, such as ``gpus``. These names must start with a
lowercase letter or an underscore, and subsequent characters can be a
lowercase letter, a digit, or an underscore. Uppercase letters are not
allowed, because certain platforms have case-insensitive environment
......@@ -1428,7 +1427,7 @@ also one cryptography chip with 4 slots.
See :prop_test:`RESOURCE_GROUPS` for a description of this property.
.. _`ctest-hardware-environment-variables`:
.. _`ctest-resource-environment-variables`:
Environment Variables
---------------------
......@@ -1449,8 +1448,8 @@ The following variables are passed to the test process:
* ``CTEST_RESOURCE_GROUP_COUNT=3``
This variable will only be defined if :manual:`ctest(1)` has been given a
``--hardware-spec-file``, or if :command:`ctest_test` has been given a
``HARDWARE_SPEC_FILE``. If no hardware specification file has been given,
``--resource-spec-file``, or if :command:`ctest_test` has been given a
``RESOURCE_SPEC_FILE``. If no resource specification file has been given,
this variable will not be defined.
.. envvar:: CTEST_RESOURCE_GROUP_<num>
......@@ -1493,7 +1492,7 @@ The following variables are passed to the test process:
the names of resource types may not clash in a case-insensitive environment.
Because of this, for the sake of simplicity, all resource types must be
listed in all lowercase in the
:ref:`hardware specification file <ctest-hardware-specification-file>` and
:ref:`resource specification file <ctest-resource-specification-file>` and
in the :prop_test:`RESOURCE_GROUPS` property, and they are converted to all
uppercase in the ``CTEST_RESOURCE_GROUP_<num>_<resource-type>`` environment
variable.
......
......@@ -2,8 +2,8 @@ RESOURCE_GROUPS
---------------
Specify resources required by a test, grouped in a way that is meaningful to
the test. See :ref:`hardware allocation <ctest-hardware-allocation>`
for more information on how this property integrates into the CTest hardware
the test. See :ref:`resource allocation <ctest-resource-allocation>`
for more information on how this property integrates into the CTest resource
allocation feature.
The ``RESOURCE_GROUPS`` property is a :ref:`semicolon-separated list <CMake
......@@ -16,7 +16,7 @@ specifying the number of slots required on one resource of the given type.
The ``RESOURCE_GROUPS`` property tells CTest what resources a test expects
to use grouped in a way meaningful to the test. The test itself must read
the :ref:`environment variables <ctest-hardware-environment-variables>` to
the :ref:`environment variables <ctest-resource-environment-variables>` to
determine which resources have been allocated to each group. For example,
each group may correspond to a process the test will spawn when executed.
......@@ -39,7 +39,7 @@ single cryptography chip. In total, 3 resource groups are specified for this
test, each with its own unique requirements.
When CTest sets the :ref:`environment variables
<ctest-hardware-environment-variables>` for a test, it assigns a group number
<ctest-resource-environment-variables>` for a test, it assigns a group number
based on the group description, starting at 0 on the left and the number of
groups minus 1 on the right. For example, in the example above, the two
groups in the first description would have IDs of 0 and 1, and the single
......
......@@ -190,8 +190,8 @@ Autogen
CTest
-----
* :manual:`ctest(1)` now has the ability to serialize tests based on hardware
requirements for each test. See :ref:`ctest-hardware-allocation` for
* :manual:`ctest(1)` now has the ability to schedule tests based on resource
requirements for each test. See :ref:`ctest-resource-allocation` for
details.
* A new test property, :prop_test:`SKIP_REGULAR_EXPRESSION`, has been added.
......
......@@ -195,7 +195,7 @@ bool cmCTestMultiProcessHandler::StartTestProcess(int test)
this->AllocateResources(test);
if (!this->TestsHaveSufficientHardware[test]) {
testRun->StartFailure("Insufficient hardware");
testRun->StartFailure("Insufficient resources");
this->FinishTestProcess(testRun, false);
return false;
}
......
......@@ -32,7 +32,7 @@ void cmCTestTestCommand::BindArguments()
this->Bind("SCHEDULE_RANDOM"_s, this->ScheduleRandom);
this->Bind("STOP_TIME"_s, this->StopTime);
this->Bind("TEST_LOAD"_s, this->TestLoad);
this->Bind("HARDWARE_SPEC_FILE"_s, this->HardwareSpecFile);
this->Bind("RESOURCE_SPEC_FILE"_s, this->HardwareSpecFile);
}
cmCTestGenericHandler* cmCTestTestCommand::InitializeHandler()
......@@ -89,7 +89,7 @@ cmCTestGenericHandler* cmCTestTestCommand::InitializeHandler()
handler->SetOption("ScheduleRandom", this->ScheduleRandom.c_str());
}
if (!this->HardwareSpecFile.empty()) {
handler->SetOption("HardwareSpecFile", this->HardwareSpecFile.c_str());
handler->SetOption("ResourceSpecFile", this->HardwareSpecFile.c_str());
}
if (!this->StopTime.empty()) {
this->CTest->SetStopTime(this->StopTime);
......
......@@ -510,12 +510,12 @@ bool cmCTestTestHandler::ProcessOptions()
}
this->SetRerunFailed(cmIsOn(this->GetOption("RerunFailed")));
val = this->GetOption("HardwareSpecFile");
val = this->GetOption("ResourceSpecFile");
if (val) {
this->UseHardwareSpec = true;
if (!this->HardwareSpec.ReadFromJSONFile(val)) {
cmCTestLog(this->CTest, ERROR_MESSAGE,
"Could not read hardware spec file: " << val << std::endl);
"Could not read resource spec file: " << val << std::endl);
return false;
}
}
......
......@@ -2090,12 +2090,12 @@ bool cmCTest::HandleCommandLineArguments(size_t& i,
"ExcludeFixtureCleanupRegularExpression", args[i].c_str());
}
if (this->CheckArgument(arg, "--hardware-spec-file") &&
if (this->CheckArgument(arg, "--resource-spec-file") &&
i < args.size() - 1) {
i++;
this->GetTestHandler()->SetPersistentOption("HardwareSpecFile",
this->GetTestHandler()->SetPersistentOption("ResourceSpecFile",
args[i].c_str());
this->GetMemCheckHandler()->SetPersistentOption("HardwareSpecFile",
this->GetMemCheckHandler()->SetPersistentOption("ResourceSpecFile",
args[i].c_str());
}
......
......@@ -103,7 +103,7 @@ static const char* cmDocumentationOptions[][2] = {
"times without failing in order to pass" },
{ "--max-width <width>", "Set the max width for a test name to output" },
{ "--interactive-debug-mode [0|1]", "Set the interactive mode to 0 or 1." },
{ "--hardware-spec-file <file>", "Set the hardware spec file to use." },
{ "--resource-spec-file <file>", "Set the resource spec file to use." },
{ "--no-label-summary", "Disable timing summary information for labels." },
{ "--no-subproject-summary",
"Disable timing summary information for "
......
......@@ -54,7 +54,7 @@ static int usageWrite(const char* argv0)
static int usageVerify(const char* argv0)
{
std::cout << "Usage: " << argv0
<< " verify <log-file> <hardware-spec-file> [<test-names>]"
<< " verify <log-file> <resource-spec-file> [<test-names>]"
<< std::endl;
return 1;
}
......@@ -286,7 +286,7 @@ static int doVerify(int argc, char const* const* argv)
cmCTestHardwareSpec spec;
if (!spec.ReadFromJSONFile(hwFile)) {
std::cout << "Could not read hardware spec " << hwFile << std::endl;
std::cout << "Could not read resource spec " << hwFile << std::endl;
return 1;
}
......
^Insufficient hardware
^Insufficient resources
CMake Error at [^
]*/Tests/RunCMake/CTestHardwareAllocation/notenough1-ctest-s-hw/test\.cmake:[0-9]+ \(message\):
Tests did not pass$
^Insufficient hardware
^Insufficient resources
CMake Error at [^
]*/Tests/RunCMake/CTestHardwareAllocation/notenough2-ctest-s-hw/test\.cmake:[0-9]+ \(message\):
Tests did not pass$
......@@ -15,7 +15,7 @@ ctest_configure(OPTIONS
ctest_build()
if(CTEST_HARDWARE_ALLOC_ENABLED)
set(hwspec HARDWARE_SPEC_FILE "@RunCMake_SOURCE_DIR@/hwspec.json")
set(hwspec RESOURCE_SPEC_FILE "@RunCMake_SOURCE_DIR@/hwspec.json")
endif()
ctest_test(${hwspec} RETURN_VALUE retval PARALLEL_LEVEL ${CTEST_PARALLEL} SCHEDULE_RANDOM ${CTEST_RANDOM})
if(retval)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment