GitLab

CTest 3.9.0-rc3 under Linux/Ubuntu 14.04 runs also some un-matching tests. For example:

ctest -R AAAA_BBBB_C
Test project XXXXXXXXXXXXXXXXXXXXXXX
    Start 26: AAAA
1/5 Test #26: AAAA ..............***Failed    0.08 sec
    Start 29: AAAA_BBBB
Failed test dependencies: AAAA
2/5 Test #29: AAAA_BBBB .......***Not Run   0.00 sec
    Start 30: AAAA_BBBB_C
Failed test dependencies: AAAA_BBBB
3/5 Test #30: AAAA_BBBB_C .....***Not Run   0.00 sec
    Start 31: AAAA_BBBB_CD
Failed test dependencies: AAAA_BBBB_C
4/5 Test #31: AAAA_BBBB_CD ....***Not Run   0.00 sec
    Start 32: AAAA_BBBB_CDE
Failed test dependencies: AAAA_BBBB_CD
5/5 Test #32: AAAA_BBBB_CDE ...***Not Run   0.00 sec

0% tests passed, 5 tests failed out of 5

Label Time Summary:
XXXXXX    =   0.08 sec (5 tests)

Total Test time (real) =   1.13 sec

The following tests FAILED:
	 26 - AAAA (Failed)
	 29 - AAAA_BBBB (Not Run)
	 30 - AAAA_BBBB_C (Not Run)
	 31 - AAAA_BBBB_CD (Not Run)
	 32 - AAAA_BBBB_CDE (Not Run)
Errors while running CTest

In the example, as expected all tests fails, BUT the "-R AAAA_BBBB_C" expression should run only:

• AAAA_BBBB_C
• AAAA_BBBB_CD
• AAAA_BBBB_CDE

Instead it tries also to run:

• AAAA
• AAAA_BBBB

I have seen that this behavior is enabled when I specify:

  set_tests_properties(${TEST_NAME} PROPERTIES FIXTURES_SETUP "${TEST_NAME}")
  set_tests_properties(${TEST_NAME} PROPERTIES FIXTURES_REQUIRED "${PARENT_TEST}")

I mean: the wrongly running two tests are set as "fixtures" of the running tests, but I am expecting they should not run if excluded by the "-R" expression. Btw, they runs even if "-E" expression explicitly disables them.