gtest_discover_tests should handle square brackets ([]) in GTest output
At the moment gtest_discover_tests
from GoogleTest module can't handle a test which has either ]
or [
in its name which isn't that rare a case with text processing and parametrized tests. For example, having some TEST_P(...)
which consumes ValuesIn("[]")
will produce some incorrect CTest behavior. I've read a few issues regarding that problem (#22897, #21027) and looks like in both cases the underlying issue is ignored and the end user doesn't even know that their tests are broken (partly).
I had this same issue but was fortunate enough to get a hard error which looked something like this:
<...> add_test called with incorrect number of arguments <...> set_tests_properties unknown error. <...> Errors while running CTest
So I tried different things to get rid of the error and found the NO_PRETTY_VALUES
parameter which looked like what I needed. And in fact with this parameter my tests were running fine and I forgot about the issue. Until I found out that CTest doesn't execute a whole bunch of tests. So by using NO_PRETTY_VALUES
I just silenced CTest but the underlying issue was still there.
And the issue is: square brackets received from the user (in this case output of GTest). I've tried to fix GoogleTestAddTests.cmake
but was unable to do so. I'm far from being a CMake expert but looks like fixing this issue properly is either very hard or outright impossible because too many commands are involved and [almost?] every command has issues with stray []
. So I think some workaround should be added to the module until some proper fix might be discovered.
I propose to add new parameters to the gtest_discover_tests
function which will allow a user to set substitution strings for the brackets. Something like OPEN_SB
& CLOSE_SB
which might have defaults or not. If they have defaults then everything will work albeit silently replacing what the user expect to something they might not expect. If they don't have defaults and the user tests have '[]' then the user should be warned that their tests might be broken and they should set these variables. At least in this case the user will see a proper warning/error not some cryptic errors which happened somewhere in the guts. Or, worse, no error at all.
Here how I see it:
- A user just uses the API but their tests have '[]':
gtest_discover_tests(${PROJECT_NAME})
. And we have something like this inGoogleTestAddTests.cmake
(beforestring(REPLACE "\n" ";" output "${output}")
):
string(REGEX MATCH [=[(\[|\])]=] OFFENSIVE_INPUT ${output})
if(OFFENSIVE_INPUT)
message(WARNING "Dear user, your tests are broken, enjoy!")
endif()
- Now the user is wise and uses the following version (new):
gtest_discover_tests(${PROJECT_NAME} OPEN_SB "【" CLOSE_SB "】")
.
I perfectly understand that it is far from perfect but at least it makes gtest_discover_tests
generally usable while right now it is a bomb which you never know if and when it blows up.