GoogleTest: gtest_discover_tests timeout based on directory
I believe there is a substantial issue with either CMake
itself or with the GTest
module. I have created a repository that demonstrates the error.
Please take a look here: https://gitlab.kitware.com/wsciaroni_ngc/cmake-gtest-bug.
cmake-gtest-bug
From the README.md of Background
This repo exists to demonstrate a bug in CMake or the GTest module.
I noticed that if I put a unit test executable in the same CMakeLists.txt as a library that the gtest_discover_tests
times out.
When utilityLibTests
is configured into the project, gtest_discover_tests
times out for this target. The project configuration is as follows.
Environment
Test on CMake 3.25.1 AND 3.26.4.
Both were tested on Windows using Visual Studio Build Tools 2019
.
Producing The Bug
I have made the buggy behavior configurable. So, the first way I demonstrate should work, then turning on an option, you will see the failure.
Working
# Notice how this works
cmake -B build -S .
cmake --build build
Broken
# Notice how this fails at build time
cmake -B build -S . -DINCLUDE_BROKEN=ON
cmake --build build
The Difference
Take a look at libraries/utilitylib/CMakeLists.txt
.
There you will find a block of code that is wrapped in an if(BUILD_TESTING AND INCLUDE_BROKEN)
.
This block is identical to the block found in tests/CMakeLists.txt
with exception of the INCLUDE_BROKEN
guard and the TEST_EXECUTABLE_NAME
.
For some reason, this configuration which works in the tests/CMakeLists.txt
FAILS to work in the libraries/utilitylib/CMakeLists.txt
.
The Error
PS C:\dev\cmake-gtest-bug> cmake --build build
Microsoft (R) Build Engine version 16.10.2+857e5a733 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
gmock.vcxproj -> C:\dev\cmake-gtest-bug\build\lib\Debug\gmock.lib
gmock_main.vcxproj -> C:\dev\cmake-gtest-bug\build\lib\Debug\gmock_main.lib
gtest.vcxproj -> C:\dev\cmake-gtest-bug\build\lib\Debug\gtest.lib
gtest_main.vcxproj -> C:\dev\cmake-gtest-bug\build\lib\Debug\gtest_main.lib
standaloneTests.vcxproj -> C:\dev\cmake-gtest-bug\build\tests\Debug\standaloneTests.exe
utilityLibTests.vcxproj -> C:\dev\cmake-gtest-bug\build\libraries\utilitylib\Debug\utilityLibTests.exe
CMake Error at C:/Program Files/CMake/share/cmake-3.25/Modules/GoogleTestAddTests.cmake:112 (message):
Error running test executable.
Path: 'C:/dev/cmake-gtest-bug/build/libraries/utilitylib/Debug/utilityLibTests.exe'
Result: Process terminated due to timeout
Output:
Call Stack (most recent call first):
C:/Program Files/CMake/share/cmake-3.25/Modules/GoogleTestAddTests.cmake:225 (gtest_discover_tests_impl)
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: The command "setlocal [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: "C:\Program Files\CMake\bin\cmake.exe" -D TEST_TARGET=utilityLibTests -D TEST_EXECUTABLE=C:/dev/cmake-gtest-bug/build/libraries/utilitylib/Debug/utilityLibTests.exe -D TEST_EXECUTOR= -D TEST_WORKING_DIR=C:/dev/cmake-gtest-bug/build/libraries/utilitylib -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D TEST_FILTER= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=utilityLibTests_TESTS -D CTEST_FILE=C:/dev/cmake-gtest-bug/build/libraries/utilitylib/utilityLibTests[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=5 -D TEST_XML_OUTPUT_DIR= -P "C:/Program Files/CMake/share/cmake-3.25/Modules/GoogleTestAddTests.cmake" [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: if %errorlevel% neq 0 goto :cmEnd [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: :cmEnd [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: :cmErrorLevel [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: exit /b %1 [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: :cmDone [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: if %errorlevel% neq 0 goto :VCEnd [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(155,5): error MSB3073: :VCEnd" exited with code 1. [C:\dev\cmake-gtest-bug\build\libraries\utilitylib\utilityLibTests.vcxproj]
Building Custom Rule C:/dev/cmake-gtest-bug/libraries/utilitylib/CMakeLists.txt
utilitylib.vcxproj -> C:\dev\cmake-gtest-bug\build\libraries\utilitylib\Debug\utilitylib.lib
PS C:\dev\cmake-gtest-bug>