1. 22 Feb, 2021 1 commit
  2. 03 Dec, 2020 1 commit
  3. 26 Aug, 2020 1 commit
  4. 01 Jul, 2020 1 commit
  5. 24 Jun, 2020 1 commit
  6. 05 May, 2020 1 commit
    • Stefan Floeren's avatar
      GoogleTestAddTests: Fix output processing · 839a1010
      Stefan Floeren authored
      The function gtest_discover_tests calls the passed test executable with
      the parameter --gtest_list_tests and parses the output to find all
      tests.
      
      In case of value-parameterized tests ([1]), the test values are included
      in the output. While test names are alphanumeric, the values can contain
      arbitrary content.
      
      First, the output is separated into lines with `foreach`. Included
      semi-colons breaks this and need to get escaped.
      
      Afterwards, the testname is passed on to the `add_command` helper. This
      helper was converted into a macro in commit dac20144 (GoogleTest:
      Optimize gtest_discover_tests, 2020-02-18). As a macro, its arguments
      are re-evaluated. Therefore we need to escape `\`, `;` and to prevent
      unwanted variable expansion `$`.
      
      Fixes: #20661
      
      [1] <https://github.com/google/googletest/blob/0eea2e9/googletest/docs/advanced.md#value-parameterized-tests>
      839a1010
  7. 10 Apr, 2020 1 commit
  8. 27 Mar, 2020 2 commits
    • Ryan Thornton's avatar
      GoogleTest: Add tests for MultiConfig discovery in PRE_TEST mode · 3b4838b5
      Ryan Thornton authored
      PRE_TEST makes it possible to properly distinguish between test cases
      that exist only in certain configurations.
      
      In the new test scenario, debug tests are disabled in release builds,
      and release tests are disabled in debug builds
      when a multi config generator is used.
      
      Note, this is a bit of a hack and *only* works for PRE_TEST mode.
      
      POST_BUILD makes no attempt to get this right. It preserves the status quo
      and you obtain the tests that were last discovered.
      
      See further discussion in !4078
      
      Ideally, the POST_BUILD behavior could be fixed
      by using generator expressions in OUTPUT and BYPRODUCT expressions.
      
      Then you could do something like:
      
          set(ctest_include_file "${ctest_file_base}_include-$<CONFIG>.cmake")
          set(ctest_tests_file "${ctest_file_base}_tests-$<CONFIG>.cmake")
      
      Once #12877
      
       lands, maybe this can be revisited.
      Co-authored-by: Ryan Thornton's avatarRyan Thornton <ThorntonRyan@JohnDeere.com>
      Co-authored-by: default avatarKevin Puetz <PuetzKevinA@JohnDeere.com>
      3b4838b5
    • Ryan Thornton's avatar
      GoogleTest: Parameterize tests to check PRE_TEST/POST_BUILD discovery mode · 1ba4cb56
      Ryan Thornton authored
      
      
      Now, the unit tests are ran twice -- once with POST_BUILD (i.e. default mode)
      and again with PRE_TEST (i.e. new discovery mode).
      
      Both modes of setting gtest discovery mode are also tested:
      1. Using the global override (i.e. CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE)
      2. Explicitly passing DISCOVERY_MODE in calls to gtest_discover_tests (in GoogleTestDiscoveryTimeout.cmake)
      
      The goal is to show that the new PRE_TEST discovery mode does not break existing behavior
      (i.e. should not break POST_BUILD mode) and should also pass the same tests
      in the same way.
      
      A few non trivial implementation details worth noting:
      
      1. Refactoring discovery_timeout_test into own project
      
      Originally, I tried doing:
      
      ```
      run_GoogleTest(POST_BUILD)
      run_GoogleTest(PRE_TEST)
      ```
      
      Without changing the internal structure of run_GoogleTest.
      
      But since discovery_timeout_test is part of the same project as the other tests,
      and CTest include files always get evaluated and that's where test discovery occurs,
      this means every other test now notices the timeout problem when running in PRE_TEST mode.
      
      As a result, keeping the existing test structure meant that each existing test
      (and any new test) would need to have its own PRE_TEST / POST_BUILD variant for stderr and stdout
      in order to handle the case where discovery_timeout_test timed out.
      
      This exponential increase in test output files introduced unnecessary complexity
      and made it more cumbersome to work on test cases.
      
      Why should an unrelated test case care about discovery_timeout_test?
      
      So, to fix that issue, the tests were broken apart into two main groups:
      1. run_GoogleTest_discovery_timeout (the test dealing with discovery_timeout_test)
      2. run_GoogleTest (everything else)
      
      This isolates the PRE_TEST / POST_BUILD timeout variants to a single test case.
      
      And the other test cases remain unchanged -- further driving home the point that
      DISCOVERY_MODE shouldn't change existing behavior.
      
      2. Different number of PRE_TEST / POST_BUILD file variants
      
      On the PRE_TEST path, different build systems / compilers (i.e. MSBuild and ninja/gcc)
      produces different build output when building discovery_timeout_test,
      but we don't actually care what it is, just as long as it builds
      successfully.
      
      This the fundamental difference in behavior between POST_BUILD (which would have failed)
      and PRE_TEST (which doesn't) and is the reason why we don't need
      a GoogleTest-discovery-build-result.txt or GoogleTest-discovery-build-stdout.txt
      
      3. Fix flaky discovery timeout test
      
      The test expects to see:
      
      > Output:
      >  timeout
      >    case.
      
      But sometimes, the test would only produce:
      
      > Output:
      >  timout
      
      In certain environments, specifically when built with OpenWatcom 1.4,
      and while the build server was under heavy load (i.e. running many tests in parallel),
      std::endl behaves inconsistently and doesn't completely
      flush std::cout when the program is terminated due to timeout.
      
      This results in inconsistent test failures because the actual output
      doesn't fully match what's expected.
      
      At first we tried adding an additional:
          std::cout << std::flush
      
      That didn't work. But using C-style printf() and fflush() appears to do
      the trick:
      
      > This time I managed to get on the machine while it was still busy doing other nightly builds
      > and could reproduce the problem reliably. With that I was finally able to find a fix.
      > It turns out my earlier hypothesis that C++ stream flushing was not working on the old compiler was correct,
      > but even .flush() is not enough.
      > I changed it to use C-style printf() and fflush() and now the test passes on that build.
      > -- Brad King <brad.king@kitware.com>
      Co-authored-by: Ryan Thornton's avatarRyan Thornton <ThorntonRyan@JohnDeere.com>
      Co-authored-by: default avatarKevin Puetz <PuetzKevinA@JohnDeere.com>
      1ba4cb56
  9. 13 Mar, 2020 1 commit
  10. 11 Mar, 2020 1 commit
    • Ryan Thornton's avatar
      GoogleTest: Add missing test case for gtest_discover_tests failure · 2c9680ee
      Ryan Thornton authored
      Discovery timeout test needs split out into two components:
      - build
      - test
      
      Building the project should result in a failure due to the timeout
      when executing gtest discovery as a post build step.
      
      Likewise, if you ran CTest after this build failure,
      we should *also* detect that the discovery_timeout_test has not been built
      because no discovery_timeout_test[1]_tests.cmake has been created
      due to the test discovery failure (caused by the timeout).
      2c9680ee
  11. 01 Jun, 2018 1 commit
    • Kitware Robot's avatar
      Revise C++ coding style using clang-format-6.0 · d7204e64
      Kitware Robot authored
      Run the `clang-format.bash` script to update all our C and C++ code to a
      new style defined by `.clang-format`.  Use `clang-format` version 6.0.
      
      * If you reached this commit for a line in `git blame`, re-run the blame
        operation starting at the parent of this commit to see older history
        for the content.
      
      * See the parent commit for instructions to rebase a change across this
        style transition commit.
      d7204e64
  12. 15 Mar, 2018 1 commit
    • Craig Scott's avatar
      GoogleTest: Rename TIMEOUT parameter to avoid clash · 96fdde26
      Craig Scott authored
      In gtest_discover_tests(), the TIMEOUT keyword was making it
      impossible to set the TIMEOUT test property via the PROPERTIES
      keyword. This would be a frequent case, but it doesn't complain
      and instead silently does something different to what would
      normally be expected. The TIMEOUT keyword has been renamed
      to DISCOVERY_TIMEOUT, thereby removing the clash.
      
      This is a breaking change. 3.10.1 and 3.10.2 were the only versions
      that supported the TIMEOUT keyword and uses of it were likely
      not working as intended.
      
      Fixes: #17801
      96fdde26
  13. 29 Dec, 2017 1 commit
  14. 06 Dec, 2017 2 commits
    • Matthew Woehlke's avatar
      GoogleTest: Add test for missing test executable · 935848a8
      Matthew Woehlke authored
      Add a test to verify expected behavior when a test executable on which
      gtest_discover_tests is invoked was not built. (The test for the newly
      added timeout conveniently sets up this situation, so this new test was
      almost trivial to add.)
      935848a8
    • Matthew Woehlke's avatar
      GoogleTest: Add timeout to discovery · 29731d89
      Matthew Woehlke authored
      Add a TIMEOUT option to gtest_discover_tests. This provides a
      work-around in case a test goes out to lunch, rather than causing the
      build to simply hang. (Although this is still a problem with the user's
      project, hanging the build is not cool, especially in the case of
      automatically running CI builds. It is much preferred that the build
      should actively fail in this case, and it is trivially easy for us to
      implement that.)
      29731d89
  15. 21 Nov, 2017 1 commit
    • Matthew Woehlke's avatar
      GoogleTest: Fix multiple discovery on same target · 70f9f62d
      Matthew Woehlke authored
      According to the documentation, tests can be discovered for a target
      multiple times by using a different prefix and/or suffix to ensure name
      uniqueness. However, while this worked for gtest_add_tests, it did not
      work with gtest_discover_tests because the generated file that sets up
      the tests was named based only on the target name, and so subsequent
      discovery from the same target would clobber earlier discovery.
      
      Fix this by introducing a counter that records how many times discovery
      has been used on a target, and use this to generate unique names of the
      generated test list files.
      70f9f62d
  16. 27 Jul, 2017 1 commit
    • Matthew Woehlke's avatar
      Add dynamic test discovery for for Google Test · bfcda401
      Matthew Woehlke authored
      Add a new gtest_discover_tests function to GoogleTest.cmake,
      implementing dynamic test discovery (i.e. tests are discovered by
      actually running the test executable and asking for the list of
      available tests, which is used to dynamically declare the tests) rather
      than the source-parsing approach used by gtest_add_tests. Compared to
      the source-parsing approach, this has the advantage of being robust
      against users declaring tests in unusual ways, and much better support
      for advanced features such as parameterized tests.
      
      A unit test, modeled after the TEST_INCLUDE_DIR[S] test, is also
      included. Note that the unit test does not actually require that Google
      Test is available. The new functionality does not actually depend on
      Google Test as such; it only requires that the test executable lists
      tests in the expected format when invoked with --gtest_list_tests, which
      the unit test can fake readily.
      bfcda401