Skip to content

Makefile: Restore use of dependency scanning cache

tsecer requested to merge tsecer/cmake:master into master

Since commit 39383ef8 our VaildDeps cache of depend.internal content is supposed to avoid re-scanning dependencies of object files whose dependencies have not changed. However, this was broken by changes to cmDependsC::WriteDependencies by commit 85cea8a7. The object file path written to depend.internal was changed to a relative path, but the lookup in the ValidDeps cache of that information was not updated too. Therefore the cache is not used.

Fix the object file path used for the ValidDeps lookup to restore the original optimization.


In dependencies Scanning phase,every source file change will cause ALL files's dependency to be regenerated

In function cmDependsC::WriteDependencies(), the input obj parameter is a full file path, while the key in this->ValidDeps map is a relative file path, so the this->ValidDeps->find(obj) will never hit. moreover this will cause every input obj's dependency to be rescanned, it's quite a painful when working with a huge project.

bool cmDependsC::WriteDependencies(const std::set<std::string>& sources,
                                   const std::string& obj,
                                   std::ostream& makeDepends,
                                   std::ostream& internalDepends)
{
......
  if (this->ValidDeps != nullptr) {
    std::map<std::string, DependencyVector>::const_iterator tmpIt =
      this->ValidDeps->find(obj);
    if (tmpIt != this->ValidDeps->end()) {
      dependencies.insert(tmpIt->second.begin(), tmpIt->second.end());
      haveDeps = true;
    }
  }
......
}

in a typical incremental build scenario, the obj parameter is read from DependInfo.cmake, which use full path format. on the other hand, ValidDeps are read from depend.internal, which use a relative path format.

in a trivial demo project, the DependInfo.cmake file:

set(CMAKE_DEPENDS_CHECK_C
  "/home/tsecer/Downloads/cmake-3.11.0-rc3/dep.rescan/foo.c" "/home/tsecer/Downloads/cmake-3.11.0-rc3/dep.rescan/CMakeFiles/depregen.dir/foo.c.o"
  "/home/tsecer/Downloads/cmake-3.11.0-rc3/dep.rescan/main.c" "/home/tsecer/Downloads/cmake-3.11.0-rc3/dep.rescan/CMakeFiles/depregen.dir/main.c.o"
  )

while the depend.internal file:

CMakeFiles/depregen.dir/foo.c.o
 /home/tsecer/Downloads/cmake-3.11.0-rc3/dep.rescan/foo.c
CMakeFiles/depregen.dir/main.c.o
 /home/tsecer/Downloads/cmake-3.11.0-rc3/dep.rescan/foo.h
 /home/tsecer/Downloads/cmake-3.11.0-rc3/dep.rescan/main.c

as mentioned, the check pairs use a full obj path /home/tsecer/Downloads/cmake-3.11.0-rc3/dep.rescan/CMakeFiles/depregen.dir/foo.c.o in CMakeFiles/depregen.dir/DependInfo.cmake, but the dependency in CMakeFiles/depregen.dir/depend.internal is a relative path CMakeFiles/depregen.dir/main.c.o. this inconsistency will surely cause a dependency rescan in cmDependsC::WriteDependencies.

a more detailed description in the attach file.

cmake-dependency-scan-ineffecient.txt


Topic-rename: makefile-fix-depend-optimization

Edited by Brad King

Merge request reports