Skip to content

Ninja Generator: Performance improvements

Matthias Männich requested to merge metti/cmake:ninja-generate into master

Commit 1296a0ea (released with 3.7) might cause performance degradations for larger projects. Especially when using custom commands as an input for each compilation rule (e.g. generated headers).

For your reference in the following I am referring to Source/cmGlobalNinjaGenerator.cxx: -> cmGlobalNinjaGenerator::AppendTargetDependsClosure -> cmGlobalNinjaGenerator::ComputeTargetDependsClosure

It turned out that the mentioned commit is doing (indirectly) some redundant work that might impact performance when generating large projects.

Imagine the dependency tree of custom targets:

A
 \
  C - D - E
 /
B

For each target the transitive closure is calculated recursively, but as the TargetDependsClosures are only cached on the top most level, everything downstream has to be recalculated. I.e.

A->C->D->E
B->C->D->E

This ultimately leads to a lot of redundant calls to AppendTargetOutputs.

I could not resolve the root cause yet (still working on it), but identified ConvertToNinjaPath as the method effectively consuming ~ 25% of ou0001-Ninja-Add-test-with-example-of-deeply-dependent-cust.patchr cmake overall runtime.

This patch is a general improvement of this method (I think so), but also demonstrates the effect. In my opinion the way the intermediate results should be kept is the way to resolve this.

What is your opinion?

EDIT: Here is a patch adding a test showing the example used for this analysis: 0001-Ninja-Add-test-with-example-of-deeply-dependent-cust.patch

Topic-rename: ninja-performance

Edited by Brad King

Merge request reports