draft: CPack/DEB: parallel parameter generation
This is a rough idea of how we could potentially parallelize CPackDeb.
This works by running CPackDeb.cmake in a separate cmake -P
process, which allows us to run as many in parallel as we want.
It currently works by spawning a fixed number, then waiting on them in the same order that they are spawned. If we go with something similar to this, we would obviously want to make the amount of parallelism configurable (CPACK_THREADS
does already exist, but the docs explicitly say that that's for compression....), as well as not wait in serial to allow for more parallelism when tasks don't all take the same duration. From my testing, they very much do not all take a similar amount of time.
Concerns/work to do:
-
CMake policies-- cmake -P
andcpack
seem to have different cmake policies enabled? I needed to add CMP0012 and CMP0007 to get it to work, and it's possible I missed some that break edgecases. -
Test cases--still doesn't pass a few. -
Make parallelism configurable -
Don't wait in order
Some performance numbers for a large project on a 24-core machine: Before:
real 17m50.839s
user 15m45.538s
sys 2m56.566s
After:
real 2m48.680s
user 23m0.374s
sys 4m12.677s
How does this approach seem? Wanted to get some feedback before putting more work into this.
Issue: #23152