Hooks for reporting progress and failures in workflow presets
Some CI systems support a form of tagging within the job output to indicate sections within the overall set of steps. For example, TeamCity allows you to embed start and end section codes, and these can be nested. When viewing the logs of such jobs, sections can be expanded and collapsed, making it easier to navigate to a particular part of the build output. Gitlab supports a similar mechanism, and I suspect most other CI systems do as well. Some may also support embedding measurement or result values which can be tracked from one run to another.
When running a CMake workflow preset, there's no opportunity for the job to provide such tags in the output. CMake embeds its own rudimentary progress markers, but to my knowledge, these are not understood by any current CI system. This means the entire workflow is seen as one monolithic chunk in the output.
Related to the above, if any part of the workflow fails and the workflow halts, the process driving the workflow has no idea what stage the workflow was executing when it failed. At best, the driving process might employ some heuristics to try to guess where the failure occurred, but this is a poor way to capture something so fundamental to the job.
Both of the above problems could be solved by giving a workflow a way to call some sort of hook before and after executing each step. A CMake script would potentially fit this idea, with data being passed to it through clearly documented CMake variables. For example, a workflow preset could support a progressScript
field, the value of which is a path to a CMake script to invoke via cmake -P ...
at the start and end of each workflow step. The cmake
command line might also support a --progress-script
option to achieve the same thing. CMake variables named WORKFLOW_NAME
, STEP_NUMBER
, STEP_NAME
, STEP_TYPE
and STEP_STATE
would be defined when running the script each time (these are just ideas to spur discussion). The script could then take arbitrary action based on these values. It might log the CI-specific section start/end markers, for example. The STEP_STATE
could hold values like START
, COMPLETE
, ERROR
, CRASH
, SKIPPED
, etc. Further state-specific details might be made available in other variables if that is appropriate, but even just having the state is already highly useful.
This doesn't seem like a particularly complicated thing to implement, at least in concept, but the value is potentially quite high. Maybe IDEs might even find it useful as a way to get reliable progress on workflows, should they implement support for workflow presets (I don't know if any do yet).