CUDA: Please add warning when dropping shared libraries from a device link
Please take a look at the following example that uses CUDA_SEPARABLE_COMPILATION: CudaLibraryConstan.zip
Building this example with BUILD_SHARED_LIBS=OFF
application succeeds, and running it also works just fine.
When built with verbose output, the device linking of the target run
looks like this:
/opt/gcc/6.4.0/bin/g++ CMakeFiles/run.dir/main.cpp.o CMakeFiles/run.dir/cmake_device_link.o -o run -L/usr/local/cuda/lib64/stubs -L/usr/local/cuda/lib64 libconsumer.a libprovider.a -lcudadevrt -lcudart_static -lrt -lpthread -ldl
CMake correctly device-links the executable run
with the library targets consumer
and provider
.
Now when doing the same thing with BUILD_SHARED_LIBS=ON
, the device link for the target consumer
looks like this:
/usr/local/cuda/bin/nvcc -gencode arch=compute_61,code=compute_61 -Xcompiler=-fPIC -Wno-deprecated-gpu-targets -shared -dlink CMakeFiles/consumer.dir/SymbolConsumer.cu.o -o CMakeFiles/consumer.dir/cmake_device_link.o
and there is no device link for the target run
.
Notice how the device link does not try to link against the target provider
, although it was specified as a dependency with target_link_libraries(consumer PRIVATE provider)
.
As mentioned by @robertmaynard somewhere, this is because NVCC completely ignores shared object files during device linking (see here, and CMake therefor silently drops any references to the shared object files.
While this might seem very obvious to someone who is intricately familiar with the CUDA tooling, for me as a somewhat novice CUDA user (who only uses CUDA through CMake exclusively), this was quite a surprise. I specified to target_link_libraries
one lib to another, and it works fine by default. Only when setting BUILD_SHARED_LIBS=ON did CMake silently ignore my command.
I think this behaviour is not ideal, and there should be a warning when this happens. It should be fairly easy to detect this situation: if the target has CUDA_SEPARABLE_COMPILATION
enabled, and the target_link_libraries
RHS argument is a shared library AND contains CUDA objects, there could be a warning along the lines of: "Device-Linking to Shared Objects is not supported. Dropping shared object link dependencies. For more information, see ...".