CTest: Integrating dashboards and Github Actions
Hi CMake developers,
Thanks very much for an amazing tool. I am on the process of moving to GitHub Actions, but I want to keep using the very useful CMake dashboard [1]. I have found lots of information on how to setup GitHub Actions with CMake and even CTest - at present I've settled on lukka's scripts [2], [3], etc - but as far as I can see, these scripts do not take into account the dashboard workflows (.e.g. CDash, I believe it was called in the past).
I am not sure if my google-fu is failing me, but I haven't found any documentation on how to do this. If so, and the documentation does exist, can you please point me in the right direction and ignore the rest of this issue :-) If in fact there is no such documentation, I'll explain briefly what I think the problems are and where I need help. I am not very knowledgeable on this topic, so I'm sure I'll make a few basic mistakes but please indulge me :-)
The Travis way
To the untrained eye, it seems that the main issue is that in the past CTest scripts were expected to contain the entire workflow. You can see the previous CTest script we used in Travis here [4] (and one we still use for our nightlies, though it is in need of updating). In Travis, we simply executed the entire script [5]:
script:
<snip>
- ctest --progress --script ".ctest.cmake,configuration_type=${configuration},generator=${generator},compiler=${compiler},number_of_jobs=${JOBS},build_group=${build_group},minimal_packaging=1,boost_test_reporting=1,code_coverage=${code_coverage}";
This was not perfect. We needed had a few hacks to cope with the fact that travis had done the checkout, but CTest also wanted to do a checkout:
# For CI, we reset the build to the current commit. We already
# know that the CI provider has done the clone and we are at the
# right commit, but we want to link the CDash view to the commit
# as well so that we can refer back to github.
set(CTEST_GIT_UPDATE_CUSTOM
"${CTEST_GIT_COMMAND}" reset --hard $ENV{BUILD_COMMIT})
With this, the version control information was correctly populated on the dashboard with the right commit.
GitHub Actions
However, things have gotten even more complicated in the brave new world of GitHub Actions (and I believe a lot of the modern IaC frameworks), because the existing GitHub Actions such as lukka's are calling CMake themselves [6]:
- name: Get latest CMake
uses: lukka/get-cmake@latest
- name: Run vcpkg
uses: lukka/run-vcpkg@v10
with:
appendedCacheKey: ${{matrix.name}}${{matrix.buildtype}}
- name: Build with CMake
uses: lukka/run-cmake@v10
with:
configurePreset: ${{matrix.family}}-${{matrix.compiler}}-${{matrix.buildtype}}
buildPreset: ${{matrix.family}}-${{matrix.compiler}}-${{matrix.buildtype}}
It is thus not obvious to the layperson how to integrate the dashboard workflow with such an approach. It seems that we can't have both the CTest script controlling the workflow but also the action doing the same. So I decided to go back to first principles.
Current state
From a layperson's perspective, it seems that CTest is merely delegating to CMake to perform the configure and build steps; if so, presumably there may be a way to "retrieve" the information generated by CMake within the lukka scripts and use those for uploading? This is what I am trying to do at present.
To cut a long story short, I am experimenting with this to try to make these two worlds play nice together. If anyone has any experience on achieving this, or any projects which I could use as an example, that would be greatly appreciated. I can write a blog post on the matter to help other people with this issue as I don't think I'm the only one trying to do this :-)
Many thanks for your time, and apologies for the rambling. :-)
Marco
[1] https://my.cdash.org/index.php?project=MASD+Project+-+Dogen
[2] https://github.com/lukka/get-cmake
[3] https://github.com/lukka/run-cmake
[4] https://github.com/MASD-Project/dogen/blob/master/CTestConfig.cmake
[5] https://github.com/MASD-Project/dogen/blob/master/.travis.yml
[6] https://github.com/MASD-Project/dogen/blob/master/.github/workflows/build-vcpkg.yml