FindProtobuf: Migration to upstream ProtobufConfig.cmake

Protobuf now provides a CMake Package Configuration file for find_package(Protobuf CONFIG), but only when it is built with CMake. The FindProtobuf module prevents that from being used directly with find_package(Protobuf). Ideally the latter should work without a find module, so !8045 (closed) proposes a policy to remove the FindProtobuf module and always rely on ProtobufConfig.cmake. However, discussion in that MR determined that the ecosystem is not yet ready.

In order for find_package(Protobuf) to be reliable without the find module, Protobuf must provide a CMake package configuration file. That means if Protobuf provides a build system other than CMake, it still needs to provide a hand-generated ProtobufConfig.cmake.

This issue tracks progress.

Cc: @a-andre @ferdnyc

mentioned in merge request !8045 (closed)

According to a member of the protobuf project

The only supported mechanism for system installation in 22.x and beyond will be cmake.

Is this sufficient to remove FindProtobuf.cmake from CMake, or has protobuf's bazel build system, which also can be used to build locally, to provide the ProtobufConfig.cmake, too?

If it is sufficient, can the policy be added when Protobuf 22.x is released, or should we wait for major distributions to ship this (or a later) version?

Great. IMO we should wait until the LTS versions of major distributions provide ProtobufConfig.cmake.

Which, realistically, means at least:

• Ubuntu 24.04 [April 2024]
• RHEL 10 [no set date; RHEL 9 was just released in May 2022, unusually soon after RHEL 8 (May 2019) succeeded RHEL 7 (June 2014), so... roughly May 2025 — May 2027? ¯\_(ツ)_/¯]
• Debian 12 [no date set, but imminent in the first half of 2023; might already be too late for this one, as it's entered "testing" and package freeze started yesterday]
• or Debian 13 [no planning even begun yet, but realistically a release comes every ~2 years, so roughly mid-2025]

But Protobuf definitely needs to release with this feature ASAP, because it may take time for distro packagers to upgrade if they need to re-write their packaging scripts to switch from autotools to CMake. And once Ubuntu decides what Protobuf release goes into 24.04, they're stuck with that one until 2029.

Plus, it'd be good to at least see CMake-based protobuf builds with ProtobufConfig.cmake configs show up in Fedora 38 and Ubuntu 23.04, for starters. Both of those start their development process soon.

We do have some options to use ProtobufConfig.cmake when available but fall back to the FindProtobuf module otherwise:

• Teach FindProtobuf to find_package(Protobuf CONFIG) first and fall back to the rest of its logic if that is not found. Several other find modules do this, including Boost, CURL, DCMTK, GLEW, and HDF5. If the package configuration file does not provide exactly the same set of imported targets and variables as the find module, some compatibility logic may be needed.

• Or, add a policy to pretend that CMAKE_FIND_PACKAGE_PREFER_CONFIG is ON for find_package(Protobuf) calls. This will cause the search for ProtobufConfig.cmake to be done first, and only load FindProtobuf if not found.

Hi @brad.king,

suggestion: Instead of policy, what about a new mode for find_package conveniently called PREFER_CONFIG, extending the signature to [CONFIG|NO_MODULE|PREFER_CONFIG]

find_package(Protobuf PREFER_CONFIG)

Basically enabling CMAKE_FIND_PACKAGE_PREFER_CONFIG for this one find_package call only.

@haraldF it shouldn't be up to the project's find_package call to handle this. This issue is about finding a general solution that can work for existing project releases to find new protobuf versions.

If you want to work around this issue in your own project, you can do this already in CMake 3.25 and above:

block()
  set(CMAKE_FIND_PACKAGE_PREFER_CONFIG 1)
  find_package(Protobuf)
endblock()

mentioned in merge request !8215 (closed)

I debugged a similar issue in https://github.com/gazebosim/gz-msgs/pull/346 . Unfortunatly I do not fully understand what could be the best change to do upstream in CMake, but here are some interesting notes:

• With Protobuf >= 4.23.0 (23.0) the target returned FindProtobuf do not work out of the box, as they do not link the required abseil targets
• The version policy used by protobuf-config.cmake and FindProtobuf.cmake is different. For protobuf-config.cmake it is effectively equivalen to SameMajorVersion (see https://github.com/protocolbuffers/protobuf/blob/v23.2/cmake/protobuf-config-version.cmake.in#L21), while I am not sure how FindProtobuf.cmake version check works, but it is seems equivalent to AnyNewerVersion.

mentioned in issue #25011 (closed)

mentioned in issue #25079

When using find_package(Protobuf CONFIG) the cmake function protobuf_generate_cpp isn't available, too. So first one should switch to the protobuf_generate function that is also available in the upstream package.

But if you have to support both versions it still gets complicated

switch to the protobuf_generate function

example:

before

include(FindProtobuf)
target_include_directories(some_target PRIVATE ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_BINARY_DIR}/protocol)
target_link_libraries(some_target PRIVATE ${Protobuf_LIBRARIES})
protobuf_generate_cpp(
    PROTO_SRC
    PROTO_HDR
    src/protocol/Proto1.proto
    src/protocol/Proto2.proto
)
target_sources(some_target PRIVATE ${PROTO_SRC} ${PROTO_HDR})
target_link_libraries(some_target PRIVATE protobuf::libprotobuf)

after

find_package(Protobuf CONFIG REQUIRED)
protobuf_generate(
    TARGET some_target
    IMPORT_DIRS src/protocol
    PROTOS
    src/protocol/Proto1.proto
    src/protocol/Proto2.proto
)
target_link_libraries(some_target PRIVATE protobuf::libprotobuf)

see also

• https://github.com/protocolbuffers/protobuf/blob/main/docs/cmake_protobuf_generate.md
• https://discourse.cmake.org/t/protobuf-generate-cpp-not-found/7053
• https://stackoverflow.com/questions/52533396/cmake-cant-find-protobuf-protobuf-generate-cpp

In https://github.com/protocolbuffers/protobuf/issues/14576 we are discussing on how to add in ProtobufConfig.cmake one feature present when using FindProtobuf.cmake, i.e. support for cross-compilation via the Protobuf_PROTOC_EXECUTABLE CMake variable, opinions are welcome.

find_package(Protbuf CONFIG)

typo: Protbuf -> Protobuf

changed the description