GoogleTest.cmake 17.8 KB
Newer Older
1 2 3 4 5 6 7
# Distributed under the OSI-approved BSD 3-Clause License.  See accompanying
# file Copyright.txt or https://cmake.org/licensing for details.

#[=======================================================================[.rst:
GoogleTest
----------

8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
This module defines functions to help use the Google Test infrastructure.  Two
mechanisms for adding tests are provided. :command:`gtest_add_tests` has been
around for some time, originally via ``find_package(GTest)``.
:command:`gtest_discover_tests` was introduced in CMake 3.10.

The (older) :command:`gtest_add_tests` scans source files to identify tests.
This is usually effective, with some caveats, including in cross-compiling
environments, and makes setting additional properties on tests more convenient.
However, its handling of parameterized tests is less comprehensive, and it
requires re-running CMake to detect changes to the list of tests.

The (newer) :command:`gtest_discover_tests` discovers tests by asking the
compiled test executable to enumerate its tests.  This is more robust and
provides better handling of parameterized tests, and does not require CMake
to be re-run when tests change.  However, it may not work in a cross-compiling
environment, and setting test properties is less convenient.

More details can be found in the documentation of the respective functions.

Both commands are intended to replace use of :command:`add_test` to register
tests, and will create a separate CTest test for each Google Test test case.
Note that this is in some cases less efficient, as common set-up and tear-down
logic cannot be shared by multiple test cases executing in the same instance.
However, it provides more fine-grained pass/fail information to CTest, which is
usually considered as more beneficial.  By default, the CTest test name is the
same as the Google Test name (i.e. ``suite.testcase``); see also
``TEST_PREFIX`` and ``TEST_SUFFIX``.
35 36 37

.. command:: gtest_add_tests

38 39
  Automatically add tests with CTest by scanning source code for Google Test
  macros::
40

41 42 43 44 45 46 47 48 49
    gtest_add_tests(TARGET target
                    [SOURCES src1...]
                    [EXTRA_ARGS arg1...]
                    [WORKING_DIRECTORY dir]
                    [TEST_PREFIX prefix]
                    [TEST_SUFFIX suffix]
                    [SKIP_DEPENDENCY]
                    [TEST_LIST outVar]
    )
50

51 52 53 54 55 56 57 58 59 60 61 62
  ``gtest_add_tests`` attempts to identify tests by scanning source files.
  Although this is generally effective, it uses only a basic regular expression
  match, which can be defeated by atypical test declarations, and is unable to
  fully "split" parameterized tests.  Additionally, it requires that CMake be
  re-run to discover any newly added, removed or renamed tests (by default,
  this means that CMake is re-run when any test source file is changed, but see
  ``SKIP_DEPENDENCY``).  However, it has the advantage of declaring tests at
  CMake time, which somewhat simplifies setting additional properties on tests,
  and always works in a cross-compiling environment.

  The options are:

63
  ``TARGET target``
64 65 66
    Specifies the Google Test executable, which must be a known CMake
    executable target.  CMake will substitute the location of the built
    executable when running the test.
67

68
  ``SOURCES src1...``
69
    When provided, only the listed files will be scanned for test cases.  If
70 71 72 73 74 75 76
    this option is not given, the :prop_tgt:`SOURCES` property of the
    specified ``target`` will be used to obtain the list of sources.

  ``EXTRA_ARGS arg1...``
    Any extra arguments to pass on the command line to each test case.

  ``WORKING_DIRECTORY dir``
77
    Specifies the directory in which to run the discovered test cases.  If this
78 79 80
    option is not provided, the current binary directory is used.

  ``TEST_PREFIX prefix``
81 82 83
    Specifies a ``prefix`` to be prepended to the name of each discovered test
    case.  This can be useful when the same source files are being used in
    multiple calls to ``gtest_add_test()`` but with different ``EXTRA_ARGS``.
84 85 86

  ``TEST_SUFFIX suffix``
    Similar to ``TEST_PREFIX`` except the ``suffix`` is appended to the name of
87 88
    every discovered test case.  Both ``TEST_PREFIX`` and ``TEST_SUFFIX`` may
    be specified.
89 90 91

  ``SKIP_DEPENDENCY``
    Normally, the function creates a dependency which will cause CMake to be
92 93
    re-run if any of the sources being scanned are changed.  This is to ensure
    that the list of discovered tests is updated.  If this behavior is not
94 95 96 97 98
    desired (as may be the case while actually writing the test cases), this
    option can be used to prevent the dependency from being added.

  ``TEST_LIST outVar``
    The variable named by ``outVar`` will be populated in the calling scope
99 100
    with the list of discovered test cases.  This allows the caller to do
    things like manipulate test properties of the discovered tests.
101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117

  .. code-block:: cmake

    include(GoogleTest)
    add_executable(FooTest FooUnitTest.cxx)
    gtest_add_tests(TARGET      FooTest
                    TEST_SUFFIX .noArgs
                    TEST_LIST   noArgsTests
    )
    gtest_add_tests(TARGET      FooTest
                    EXTRA_ARGS  --someArg someValue
                    TEST_SUFFIX .withArgs
                    TEST_LIST   withArgsTests
    )
    set_tests_properties(${noArgsTests}   PROPERTIES TIMEOUT 10)
    set_tests_properties(${withArgsTests} PROPERTIES TIMEOUT 20)

118
  For backward compatibility, the following form is also supported::
119 120 121 122 123 124

    gtest_add_tests(exe args files...)

  ``exe``
    The path to the test executable or the name of a CMake target.
  ``args``
125 126 127
    A ;-list of extra arguments to be passed to executable.  The entire
    list must be passed as a single argument.  Enclose it in quotes,
    or pass ``""`` for no arguments.
128
  ``files...``
129
    A list of source files to search for tests and test fixtures.
130
    Alternatively, use ``AUTO`` to specify that ``exe`` is the name
131 132
    of a CMake executable target whose sources should be scanned.

133
  .. code-block:: cmake
134

135 136 137 138
    include(GoogleTest)
    set(FooTestArgs --foo 1 --bar 2)
    add_executable(FooTest FooUnitTest.cxx)
    gtest_add_tests(FooTest "${FooTestArgs}" AUTO)
139

140 141 142 143 144 145 146 147 148 149 150 151 152
.. command:: gtest_discover_tests

  Automatically add tests with CTest by querying the compiled test executable
  for available tests::

    gtest_discover_tests(target
                         [EXTRA_ARGS arg1...]
                         [WORKING_DIRECTORY dir]
                         [TEST_PREFIX prefix]
                         [TEST_SUFFIX suffix]
                         [NO_PRETTY_TYPES] [NO_PRETTY_VALUES]
                         [PROPERTIES name1 value1...]
                         [TEST_LIST var]
153
                         [DISCOVERY_TIMEOUT seconds]
154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220
    )

  ``gtest_discover_tests`` sets up a post-build command on the test executable
  that generates the list of tests by parsing the output from running the test
  with the ``--gtest_list_tests`` argument.  Compared to the source parsing
  approach of :command:`gtest_add_tests`, this ensures that the full list of
  tests, including instantiations of parameterized tests, is obtained.  Since
  test discovery occurs at build time, it is not necessary to re-run CMake when
  the list of tests changes.
  However, it requires that :prop_tgt:`CROSSCOMPILING_EMULATOR` is properly set
  in order to function in a cross-compiling environment.

  Additionally, setting properties on tests is somewhat less convenient, since
  the tests are not available at CMake time.  Additional test properties may be
  assigned to the set of tests as a whole using the ``PROPERTIES`` option.  If
  more fine-grained test control is needed, custom content may be provided
  through an external CTest script using the :prop_dir:`TEST_INCLUDE_FILES`
  directory property.  The set of discovered tests is made accessible to such a
  script via the ``<target>_TESTS`` variable.

  The options are:

  ``target``
    Specifies the Google Test executable, which must be a known CMake
    executable target.  CMake will substitute the location of the built
    executable when running the test.

  ``EXTRA_ARGS arg1...``
    Any extra arguments to pass on the command line to each test case.

  ``WORKING_DIRECTORY dir``
    Specifies the directory in which to run the discovered test cases.  If this
    option is not provided, the current binary directory is used.

  ``TEST_PREFIX prefix``
    Specifies a ``prefix`` to be prepended to the name of each discovered test
    case.  This can be useful when the same test executable is being used in
    multiple calls to ``gtest_discover_tests()`` but with different
    ``EXTRA_ARGS``.

  ``TEST_SUFFIX suffix``
    Similar to ``TEST_PREFIX`` except the ``suffix`` is appended to the name of
    every discovered test case.  Both ``TEST_PREFIX`` and ``TEST_SUFFIX`` may
    be specified.

  ``NO_PRETTY_TYPES``
    By default, the type index of type-parameterized tests is replaced by the
    actual type name in the CTest test name.  If this behavior is undesirable
    (e.g. because the type names are unwieldy), this option will suppress this
    behavior.

  ``NO_PRETTY_VALUES``
    By default, the value index of value-parameterized tests is replaced by the
    actual value in the CTest test name.  If this behavior is undesirable
    (e.g. because the value strings are unwieldy), this option will suppress
    this behavior.

  ``PROPERTIES name1 value1...``
    Specifies additional properties to be set on all tests discovered by this
    invocation of ``gtest_discover_tests``.

  ``TEST_LIST var``
    Make the list of tests available in the variable ``var``, rather than the
    default ``<target>_TESTS``.  This can be useful when the same test
    executable is being used in multiple calls to ``gtest_discover_tests()``.
    Note that this variable is only available in CTest.

221
  ``DISCOVERY_TIMEOUT num``
222 223 224 225 226 227 228
    Specifies how long (in seconds) CMake will wait for the test to enumerate
    available tests.  If the test takes longer than this, discovery (and your
    build) will fail.  Most test executables will enumerate their tests very
    quickly, but under some exceptional circumstances, a test may require a
    longer timeout.  The default is 5.  See also the ``TIMEOUT`` option of
    :command:`execute_process`.

229 230 231 232 233 234 235 236 237 238
    .. note::

      In CMake versions 3.10.1 and 3.10.2, this option was called ``TIMEOUT``.
      This clashed with the ``TIMEOUT`` test property, which is one of the
      common properties that would be set with the ``PROPERTIES`` keyword,
      usually leading to legal but unintended behavior.  The keyword was
      changed to ``DISCOVERY_TIMEOUT`` in CMake 3.10.3 to address this
      problem.  The ambiguous behavior of the ``TIMEOUT`` keyword in 3.10.1
      and 3.10.2 has not been preserved.

239 240
#]=======================================================================]

241 242 243 244
# Save project's policies
cmake_policy(PUSH)
cmake_policy(SET CMP0057 NEW) # if IN_LIST

245
#------------------------------------------------------------------------------
246 247 248 249
function(gtest_add_tests)

  if (ARGC LESS 1)
    message(FATAL_ERROR "No arguments supplied to gtest_add_tests()")
250
  endif()
251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298

  set(options
      SKIP_DEPENDENCY
  )
  set(oneValueArgs
      TARGET
      WORKING_DIRECTORY
      TEST_PREFIX
      TEST_SUFFIX
      TEST_LIST
  )
  set(multiValueArgs
      SOURCES
      EXTRA_ARGS
  )
  set(allKeywords ${options} ${oneValueArgs} ${multiValueArgs})

  unset(sources)
  if("${ARGV0}" IN_LIST allKeywords)
    cmake_parse_arguments(ARGS "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
    set(autoAddSources YES)
  else()
    # Non-keyword syntax, convert to keyword form
    if (ARGC LESS 3)
      message(FATAL_ERROR "gtest_add_tests() without keyword options requires at least 3 arguments")
    endif()
    set(ARGS_TARGET     "${ARGV0}")
    set(ARGS_EXTRA_ARGS "${ARGV1}")
    if(NOT "${ARGV2}" STREQUAL "AUTO")
      set(ARGS_SOURCES "${ARGV}")
      list(REMOVE_AT ARGS_SOURCES 0 1)
    endif()
  endif()

  # The non-keyword syntax allows the first argument to be an arbitrary
  # executable rather than a target if source files are also provided. In all
  # other cases, both forms require a target.
  if(NOT TARGET "${ARGS_TARGET}" AND NOT ARGS_SOURCES)
    message(FATAL_ERROR "${ARGS_TARGET} does not define an existing CMake target")
  endif()
  if(NOT ARGS_WORKING_DIRECTORY)
    unset(workDir)
  else()
    set(workDir WORKING_DIRECTORY "${ARGS_WORKING_DIRECTORY}")
  endif()

  if(NOT ARGS_SOURCES)
    get_property(ARGS_SOURCES TARGET ${ARGS_TARGET} PROPERTY SOURCES)
299
  endif()
300 301 302

  unset(testList)

303 304
  set(gtest_case_name_regex ".*\\( *([A-Za-z_0-9]+) *, *([A-Za-z_0-9]+) *\\).*")
  set(gtest_test_type_regex "(TYPED_TEST|TEST_?[FP]?)")
305 306 307 308 309

  foreach(source IN LISTS ARGS_SOURCES)
    if(NOT ARGS_SKIP_DEPENDENCY)
      set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS ${source})
    endif()
310
    file(READ "${source}" contents)
311
    string(REGEX MATCHALL "${gtest_test_type_regex} *\\(([A-Za-z_0-9 ,]+)\\)" found_tests "${contents}")
312 313 314 315 316
    foreach(hit ${found_tests})
      string(REGEX MATCH "${gtest_test_type_regex}" test_type ${hit})

      # Parameterized tests have a different signature for the filter
      if("x${test_type}" STREQUAL "xTEST_P")
317
        string(REGEX REPLACE ${gtest_case_name_regex}  "*/\\1.\\2/*" gtest_test_name ${hit})
318
      elseif("x${test_type}" STREQUAL "xTEST_F" OR "x${test_type}" STREQUAL "xTEST")
319
        string(REGEX REPLACE ${gtest_case_name_regex} "\\1.\\2" gtest_test_name ${hit})
320
      elseif("x${test_type}" STREQUAL "xTYPED_TEST")
321
        string(REGEX REPLACE ${gtest_case_name_regex} "\\1/*.\\2" gtest_test_name ${hit})
322 323 324 325
      else()
        message(WARNING "Could not parse GTest ${hit} for adding to CTest.")
        continue()
      endif()
326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360

      # Make sure tests disabled in GTest get disabled in CTest
      if(gtest_test_name MATCHES "(^|\\.)DISABLED_")
        # Add the disabled test if CMake is new enough
        # Note that this check is to allow backwards compatibility so this
        # module can be copied locally in projects to use with older CMake
        # versions
        if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.8.20170401)
          string(REGEX REPLACE
                 "(^|\\.)DISABLED_" "\\1"
                 orig_test_name "${gtest_test_name}"
          )
          set(ctest_test_name
              ${ARGS_TEST_PREFIX}${orig_test_name}${ARGS_TEST_SUFFIX}
          )
          add_test(NAME ${ctest_test_name}
                   ${workDir}
                   COMMAND ${ARGS_TARGET}
                     --gtest_also_run_disabled_tests
                     --gtest_filter=${gtest_test_name}
                     ${ARGS_EXTRA_ARGS}
          )
          set_tests_properties(${ctest_test_name} PROPERTIES DISABLED TRUE)
          list(APPEND testList ${ctest_test_name})
        endif()
      else()
        set(ctest_test_name ${ARGS_TEST_PREFIX}${gtest_test_name}${ARGS_TEST_SUFFIX})
        add_test(NAME ${ctest_test_name}
                 ${workDir}
                 COMMAND ${ARGS_TARGET}
                   --gtest_filter=${gtest_test_name}
                   ${ARGS_EXTRA_ARGS}
        )
        list(APPEND testList ${ctest_test_name})
      endif()
361 362
    endforeach()
  endforeach()
363 364 365 366 367

  if(ARGS_TEST_LIST)
    set(${ARGS_TEST_LIST} ${testList} PARENT_SCOPE)
  endif()

368
endfunction()
369 370 371 372 373 374

#------------------------------------------------------------------------------
function(gtest_discover_tests TARGET)
  cmake_parse_arguments(
    ""
    "NO_PRETTY_TYPES;NO_PRETTY_VALUES"
375
    "TEST_PREFIX;TEST_SUFFIX;WORKING_DIRECTORY;TEST_LIST;DISCOVERY_TIMEOUT"
376 377 378 379 380 381 382 383 384 385
    "EXTRA_ARGS;PROPERTIES"
    ${ARGN}
  )

  if(NOT _WORKING_DIRECTORY)
    set(_WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
  endif()
  if(NOT _TEST_LIST)
    set(_TEST_LIST ${TARGET}_TESTS)
  endif()
386 387
  if(NOT _DISCOVERY_TIMEOUT)
    set(_DISCOVERY_TIMEOUT 5)
388
  endif()
389

390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411
  get_property(
    has_counter
    TARGET ${TARGET}
    PROPERTY CTEST_DISCOVERED_TEST_COUNTER
    SET
  )
  if(has_counter)
    get_property(
      counter
      TARGET ${TARGET}
      PROPERTY CTEST_DISCOVERED_TEST_COUNTER
    )
    math(EXPR counter "${counter} + 1")
  else()
    set(counter 1)
  endif()
  set_property(
    TARGET ${TARGET}
    PROPERTY CTEST_DISCOVERED_TEST_COUNTER
    ${counter}
  )

412
  # Define rule to generate test list for aforementioned test executable
413 414 415
  set(ctest_file_base "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}[${counter}]")
  set(ctest_include_file "${ctest_file_base}_include.cmake")
  set(ctest_tests_file "${ctest_file_base}_tests.cmake")
416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435
  get_property(crosscompiling_emulator
    TARGET ${TARGET}
    PROPERTY CROSSCOMPILING_EMULATOR
  )
  add_custom_command(
    TARGET ${TARGET} POST_BUILD
    BYPRODUCTS "${ctest_tests_file}"
    COMMAND "${CMAKE_COMMAND}"
            -D "TEST_TARGET=${TARGET}"
            -D "TEST_EXECUTABLE=$<TARGET_FILE:${TARGET}>"
            -D "TEST_EXECUTOR=${crosscompiling_emulator}"
            -D "TEST_WORKING_DIR=${_WORKING_DIRECTORY}"
            -D "TEST_EXTRA_ARGS=${_EXTRA_ARGS}"
            -D "TEST_PROPERTIES=${_PROPERTIES}"
            -D "TEST_PREFIX=${_TEST_PREFIX}"
            -D "TEST_SUFFIX=${_TEST_SUFFIX}"
            -D "NO_PRETTY_TYPES=${_NO_PRETTY_TYPES}"
            -D "NO_PRETTY_VALUES=${_NO_PRETTY_VALUES}"
            -D "TEST_LIST=${_TEST_LIST}"
            -D "CTEST_FILE=${ctest_tests_file}"
436
            -D "TEST_DISCOVERY_TIMEOUT=${_DISCOVERY_TIMEOUT}"
437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460
            -P "${_GOOGLETEST_DISCOVER_TESTS_SCRIPT}"
    VERBATIM
  )

  file(WRITE "${ctest_include_file}"
    "if(EXISTS \"${ctest_tests_file}\")\n"
    "  include(\"${ctest_tests_file}\")\n"
    "else()\n"
    "  add_test(${TARGET}_NOT_BUILT ${TARGET}_NOT_BUILT)\n"
    "endif()\n"
  )

  # Add discovered tests to directory TEST_INCLUDE_FILES
  set_property(DIRECTORY
    APPEND PROPERTY TEST_INCLUDE_FILES "${ctest_include_file}"
  )

endfunction()

###############################################################################

set(_GOOGLETEST_DISCOVER_TESTS_SCRIPT
  ${CMAKE_CURRENT_LIST_DIR}/GoogleTestAddTests.cmake
)
461 462 463

# Restore project's policies
cmake_policy(POP)