CMake & Ninja fail to recompile dependent generated c++ files on protobuf source file change
Hi,
we experience an issue with CMake & Ninja Protobuf support in our project. Whenever .proto file is changed and .cc/.h files are regenerated, other generated dependent files are not re-compiled on the first ninja run. The produced linked binary is then often broken and crashes at runtime. Ninja has to be invoked for the second time to finish the build properly. Although, invoking it twice is a possible workaround, it seems to be a hack, so we'd prefer to have a proper solution.
I'm not sure whether it is a issue in CMake's ninja generator, in CMake's support for Protobuf or in Ninja itself. Below is more investigation info. I've also created a simple reproduction project: https://github.com/Cry-Pavlo/test_protobuf_ninja_rebuild
The issue was reproduced with CMake 3.16.1 and Ninja 1.9.0.
Details
Let there be two proto files: a.proto
and b.proto
. b.proto
should import and use a.proto
, which means that generated b.pb.cc
includes a.pb.h
.
Let there also be main.cpp
, which includes a.pb.h
.
Expectation is, any time file a.proto
is modified, every dependent file (main.cpp
, a.pb.cc
, b.pb.cc
) is recompiled.
Current behavior is following: when a.proto
is modified, Ninja properly regenerates a.pb.h
& a.pb.cc
, recompiles a.pb.cc.obj
and recompiles all source files which were including a.pb.h
(i.e. main.cpp
), except the generated b.pb.cc
. When invoking Ninja for the second time, it realizes that a.pb.h
is newer and finally recompiles dependent b.pb.cc.obj
.
Here is the example log with the debug output - see how it requires two invocation of Ninja to get full recompilation:
d:\dev\test_proto_ninja_failure\build>d:\bin\ninja\ninja.exe -v -d explain
ninja explain: restat of output GenProto/a.pb.h older than most recent input ../a.proto (5984568593552928 vs 5984568885972004)
ninja explain: GenProto/a.pb.h is dirty
ninja explain: CMakeFiles/TestNinjaGen.dir/main.cpp.obj is dirty
ninja explain: GenProto/a.pb.cc is dirty
ninja explain: CMakeFiles/TestNinjaGen.dir/GenProto/a.pb.cc.obj is dirty
ninja explain: TestNinjaGen.exe is dirty
[1/4] cmd.exe /C "cd /D D:\dev\test_proto_ninja_failure\build && D:\dev\test_proto_ninja_failure\protobuf\bin\protoc.exe --cpp_out GenProto -I D:/dev/test_proto_ninja_failure D:/dev/test_proto_ninja_failure/a.proto"
[2/4] C:\PROGRA~2\MICROS~3\2017\PROFES~1\VC\Tools\MSVC\1414~1.264\bin\Hostx86\x86\cl.exe /nologo /TP -IGenProto -I..\protobuf\include /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MD /O2 /Ob2 /DNDEBUG -MD /showIncludes /FoCMakeFiles\TestNinjaGen.dir\main.cpp.obj /FdCMakeFiles\TestNinjaGen.dir\ /FS -c ..\main.cpp
[3/4] C:\PROGRA~2\MICROS~3\2017\PROFES~1\VC\Tools\MSVC\1414~1.264\bin\Hostx86\x86\cl.exe /nologo /TP -IGenProto -I..\protobuf\include /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MD /O2 /Ob2 /DNDEBUG -MD /showIncludes /FoCMakeFiles\TestNinjaGen.dir\GenProto\a.pb.cc.obj /FdCMakeFiles\TestNinjaGen.dir\ /FS -c GenProto\a.pb.cc
GenProto\a.pb.cc(145): warning C4244: '=': conversion from 'google::protobuf::uint64' to 'google::protobuf::int32', possible loss of data
GenProto\a.pb.cc(152): warning C4244: '=': conversion from 'google::protobuf::uint64' to 'google::protobuf::int32', possible loss of data
[4/4] cmd.exe /C "cd . && D:\bin\cmake-3.16.1-win64-x64\bin\cmake.exe -E vs_link_exe --intdir=CMakeFiles\TestNinjaGen.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100171~1.0\x86\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100171~1.0\x86\mt.exe --manifests -- C:\PROGRA~2\MICROS~3\2017\PROFES~1\VC\Tools\MSVC\1414~1.264\bin\Hostx86\x86\link.exe /nologo CMakeFiles\TestNinjaGen.dir\main.cpp.obj CMakeFiles\TestNinjaGen.dir\GenProto\a.pb.cc.obj CMakeFiles\TestNinjaGen.dir\GenProto\b.pb.cc.obj /out:TestNinjaGen.exe /implib:TestNinjaGen.lib /pdb:TestNinjaGen.pdb /version:0.0 /machine:X86 /INCREMENTAL:NO /subsystem:console ..\protobuf\lib\libprotobuf.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
d:\dev\test_proto_ninja_failure\build>d:\bin\ninja\ninja.exe -v -d explain
ninja explain: output CMakeFiles/TestNinjaGen.dir/GenProto/b.pb.cc.obj older than most recent input genproto/a.pb.h (5984568609679100 vs 5984568918708987)
ninja explain: CMakeFiles/TestNinjaGen.dir/GenProto/b.pb.cc.obj is dirty
ninja explain: TestNinjaGen.exe is dirty
[1/2] C:\PROGRA~2\MICROS~3\2017\PROFES~1\VC\Tools\MSVC\1414~1.264\bin\Hostx86\x86\cl.exe /nologo /TP -IGenProto -I..\protobuf\include /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MD /O2 /Ob2 /DNDEBUG -MD /showIncludes /FoCMakeFiles\TestNinjaGen.dir\GenProto\b.pb.cc.obj /FdCMakeFiles\TestNinjaGen.dir\ /FS -c GenProto\b.pb.cc
GenProto\b.pb.cc(168): warning C4244: '=': conversion from 'google::protobuf::uint64' to 'google::protobuf::int32', possible loss of data
[2/2] cmd.exe /C "cd . && D:\bin\cmake-3.16.1-win64-x64\bin\cmake.exe -E vs_link_exe --intdir=CMakeFiles\TestNinjaGen.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100171~1.0\x86\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100171~1.0\x86\mt.exe --manifests -- C:\PROGRA~2\MICROS~3\2017\PROFES~1\VC\Tools\MSVC\1414~1.264\bin\Hostx86\x86\link.exe /nologo CMakeFiles\TestNinjaGen.dir\main.cpp.obj CMakeFiles\TestNinjaGen.dir\GenProto\a.pb.cc.obj CMakeFiles\TestNinjaGen.dir\GenProto\b.pb.cc.obj /out:TestNinjaGen.exe /implib:TestNinjaGen.lib /pdb:TestNinjaGen.pdb /version:0.0 /machine:X86 /INCREMENTAL:NO /subsystem:console ..\protobuf\lib\libprotobuf.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
d:\dev\test_proto_ninja_failure\build>d:\bin\ninja\ninja.exe -v -d explain
ninja: no work to do.
The ninja dependency discovery looks correct, ninja -t deps
shows:
CMakeFiles/TestNinjaGen.dir/GenProto/b.pb.cc.obj: #deps 50, deps mtime 5984568989188494 (VALID)
genproto/a.pb.h
genproto/b.pb.h
I've had a theory, that the issue is caused by some interplay of order-only and implicit dependencies. build.ninja
has following lines:
build cmake_object_order_depends_target_TestNinjaGen: phony || GenProto\a.pb.cc GenProto\a.pb.h GenProto\b.pb.cc GenProto\b.pb.h
build CMakeFiles\TestNinjaGen.dir\GenProto\b.pb.cc.obj: CXX_COMPILER__TestNinjaGen GenProto\b.pb.cc || cmake_object_order_depends_target_TestNinjaGen
b.pb.cc.obj
is order-only dependent on cmake_object_order_depends_target_TestNinjaGen
(i.e. on a.pb.h
) and also implicitly dependent on a.pb.h
(through header-file dependency discovery). Maybe order-only is more important and makes the rule for b.pb.cc.obj
to ignore timestamp somehow?
I tried to manually modify build.ninja
and remove the order-only dependency, but it doesn't help.