Control of order of building independent targets
I'm facing an issue where out of many targets in my project there are a handful that contain files that take significantly more time (~10x) than others for compilation. Currently, with Ninja generator on Linux the build time is about twice as long as with the older tool we're aiming to replace which scheduled these targets at the beginning of the build process.
After trying to find ways to influence this order and looking at CMake code I've noticed that (1) Ninja processes leaf targets in the lexer order and (2) CMake's order of emitting the targets into build.ninja is (at least partially) defined by order of iteration over cmTargets
which is an unordered_map
with name as a key.
With a quick change to the CMake code, I've identified that changing unordered_map
to a std::list
of pairs maintains the order that the targets were created in the cmake scripts and I was able to issue the 'heavy' targets first reducing build time through increasing the utilization in the parallel build. Obviously, list
is not a solution, but coming up with a data structure that maintains order of insertion and has O(1)
for find
and insert
operations would not be difficult.
On the other hand, this seems to be quite an inflexible and implicit solution to the problem. An alternative is, as a first step, to add COST
to the target properties and treat them similarly to the COST
property of tests (i.e., the higher the cost, the earlier the target is built). Then, at generation time, it becomes possible to take this cost into account when emitting the resulting build system files. For Ninja, it seems to be as simple as sorting the target list according to this cost in or after cmGlobalGenerator::CreateGeneratorTargets
.
What do you think? Would you consider such a pull request? Do you think that this goal of ordering leaf targets is attainable in other generators?
Rostislav.
PS: the next step would probably be adding COST to source files with long compilation times, but for now if that were necessary it's always possible to create an intermediate OBJECT library.