Commit b60789a7 authored by Zack Galbreath's avatar Zack Galbreath
Browse files

Help: Document CTest custom test measurements

parent 63b5ddcc
......@@ -170,3 +170,83 @@ The options are:
See also the :variable:`CTEST_CUSTOM_MAXIMUM_PASSED_TEST_OUTPUT_SIZE`
and :variable:`CTEST_CUSTOM_MAXIMUM_FAILED_TEST_OUTPUT_SIZE` variables.
.. _`Additional Test Measurements`:
Additional Test Measurements
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
CTest can parse the output of your tests for extra measurements to report
to CDash.
When run as a :ref:`Dashboard Client`, CTest will include these custom
measurements in the ``Test.xml`` file that gets uploaded to CDash.
Check the `CDash test measurement documentation
<https://github.com/Kitware/CDash/blob/master/docs/test_measurements.md>`_
for more information on the types of test measurements that CDash recognizes.
The following example demonstrates how to output a variety of custom test
measurements.
.. code-block:: c++
std::cout <<
"<DartMeasurement type=\"numeric/double\" name=\"score\">28.3</DartMeasurement>"
<< std::endl;
std::cout <<
"<DartMeasurement type=\"text/string\" name=\"color\">red</DartMeasurement>"
<< std::endl;
std::cout <<
"<DartMeasurement type=\"text/link\" name=\"CMake URL\">https://cmake.org</DartMeasurement>"
<< std::endl;
std::cout <<
"<DartMeasurement type=\"text/preformatted\" name=\"Console Output\">" <<
"line 1.\n" <<
" \033[31;1m line 2. Bold red, and indented!\033[0;0ml\n" <<
"line 3. Not bold or indented...\n" <<
"</DartMeasurement>" << std::endl;
Image Measurements
""""""""""""""""""
The following example demonstrates how to upload test images to CDash.
.. code-block:: c++
std::cout <<
"<DartMeasurementFile type=\"image/jpg\" name=\"TestImage\">" <<
"/dir/to/test_img.jpg</DartMeasurementFile>" << std::endl;
std::cout <<
"<DartMeasurementFile type=\"image/gif\" name=\"ValidImage\">" <<
"/dir/to/valid_img.gif</DartMeasurementFile>" << std::endl;
std::cout <<
"<DartMeasurementFile type=\"image/png\" name=\"AlgoResult\"> <<
"/dir/to/img.png</DartMeasurementFile>"
<< std::endl;
Images will be displayed together in an interactive comparison mode on CDash
if they are provided with two or more of the following names.
* ``TestImage``
* ``ValidImage``
* ``BaselineImage``
* ``DifferenceImage2``
By convention, ``TestImage`` is the image generated by your test, and
``ValidImage`` (or ``BaselineImage``) is basis of comparison used to determine
if the test passed or failed.
If another image name is used it will be displayed by CDash as a static image
separate from the interactive comparison UI.
Attached Files
""""""""""""""
To associate other types of files with a test, use the
:prop_test:`ATTACHED_FILES` or :prop_test:`ATTACHED_FILES_ON_FAIL` test properties.
......@@ -1095,6 +1095,8 @@ Configuration settings include:
* `CTest Script`_ variable: :variable:`CTEST_TEST_TIMEOUT`
* :module:`CTest` module variable: ``DART_TESTING_TIMEOUT``
To report extra test values to CDash, see :ref:`Additional Test Measurements`.
.. _`CTest Coverage Step`:
CTest Coverage Step
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment