Skip to content

GoogleTest: gtest_discover_tests timeout based on directory

I believe there is a substantial issue with either CMake itself or with the GTest module. I have created a repository that demonstrates the error.

Please take a look here: https://gitlab.kitware.com/wsciaroni_ngc/cmake-gtest-bug.

From the README.md of cmake-gtest-bug

Background

This repo exists to demonstrate a bug in CMake or the GTest module.

I noticed that if I put a unit test executable in the same CMakeLists.txt as a library that the gtest_discover_tests times out.

When utilityLibTests is configured into the project, gtest_discover_tests times out for this target. The project configuration is as follows.

Environment

Test on CMake 3.25.1 AND 3.26.4.

Both were tested on Windows using Visual Studio Build Tools 2019.

Producing The Bug

I have made the buggy behavior configurable. So, the first way I demonstrate should work, then turning on an option, you will see the failure.

Working

# Notice how this works
cmake -B build -S .
cmake --build build

Broken

# Notice how this fails at build time
cmake -B build -S . -DINCLUDE_BROKEN=ON
cmake --build build

The Difference

Take a look at libraries/utilitylib/CMakeLists.txt. There you will find a block of code that is wrapped in an if(BUILD_TESTING AND INCLUDE_BROKEN). This block is identical to the block found in tests/CMakeLists.txt with exception of the INCLUDE_BROKEN guard and the TEST_EXECUTABLE_NAME.

For some reason, this configuration which works in the tests/CMakeLists.txt FAILS to work in the libraries/utilitylib/CMakeLists.txt.

The Error

PS C:\dev\cmake-gtest-bug> cmake --build build
Microsoft (R) Build Engine version 16.10.2+857e5a733 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

  gmock.vcxproj -> C:\dev\cmake-gtest-bug\build\lib\Debug\gmock.lib
  gmock_main.vcxproj -> C:\dev\cmake-gtest-bug\build\lib\Debug\gmock_main.lib
  gtest.vcxproj -> C:\dev\cmake-gtest-bug\build\lib\Debug\gtest.lib
  gtest_main.vcxproj -> C:\dev\cmake-gtest-bug\build\lib\Debug\gtest_main.lib
  standaloneTests.vcxproj -> C:\dev\cmake-gtest-bug\build\tests\Debug\standaloneTests.exe
  utilityLibTests.vcxproj -> C:\dev\cmake-gtest-bug\build\libraries\utilitylib\Debug\utilityLibTests.exe
  CMake Error at C:/Program Files/CMake/share/cmake-3.25/Modules/GoogleTestAddTests.cmake:112 (message):
    Error running test executable.

      Path: 'C:/dev/cmake-gtest-bug/build/libraries/utilitylib/Debug/utilityLibTests.exe'
      Result: Process terminated due to timeout
      Output:


  Call Stack (most recent call first):
    C:/Program Files/CMake/share/cmake-3.25/Modules/GoogleTestAddTests.cmake:225 (gtest_discover_tests_impl)


C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: The command "setlocal [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: "C:\Program Files\CMake\bin\cmake.exe" -D TEST_TARGET=utilityLibTests -D TEST_EXECUTABLE=C:/dev/cmake-gtest-bug/build/libraries/utilitylib/Debug/utilityLibTests.exe -D TEST_EXECUTOR= -D TEST_WORKING_DIR=C:/dev/cmake-gtest-bug/build/libraries/utilitylib -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D TEST_FILTER= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=utilityLibTests_TESTS -D CTEST_FILE=C:/dev/cmake-gtest-bug/build/libraries/utilitylib/utilityLibTests[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=5 -D TEST_XML_OUTPUT_DIR= -P "C:/Program Files/CMake/share/cmake-3.25/Modules/GoogleTestAddTests.cmake" [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: if %errorlevel% neq 0 goto :cmEnd [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: :cmEnd [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]   
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: :cmErrorLevel [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: exit /b %1 [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: :cmDone [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]  
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: if %errorlevel% neq 0 goto :VCEnd [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: :VCEnd" exited with code 1. [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
  Building Custom Rule C:/dev/cmake-gtest-bug/libraries/utilitylib/CMakeLists.txt
  utilitylib.vcxproj -> C:\dev\cmake-gtest-bug\build\libraries\utilitylib\Debug\utilitylib.lib
PS C:\dev\cmake-gtest-bug> 
Edited by William Sciaroni
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information