libraries found by find_package are not always linked correctly in external projects
I have noticed a trend in that when I use cmake to build an external project, that the find_package function in the project's cmake correctly finds a package, libraries, includes, etc. However, when it comes to link time, the versions found by those projects are typically reverted to the systems version (linux). I'm having trouble figuring out why this is happening - it's in more than one project. I am using CMake 3.12.3
One piece of information I've found is that if cmake is passed a library that exists in the LD_LIBRARY_PATH, that library name is shortened automatically (ie /home/steve/lib/libfoo.so
becomes -lfoo
). I feel as if this is having some sort of impact, in that when it comes to link time, the linker may be trying to find -lfoo
, and for some reason, it finds -lfoo
in /usr/lib{64}
, and tries to link with it. I found a post a while back that suggested adding these libraries as imported targets, and setting their property to the found location:
find_package(ZLIB)
if(ZLIB_FOUND)
add_library(z SHARED IMPORTED)
set_property(TARGET z PROPERTY IMPORTED_LOCATION ${ZLIB_LIBRARY})
endif(ZLIB_FOUND)
I can edit the project I'm working on, such that it applies a patch to the project's CMakeLists.txt, but I feel like that is a deficiency in the build process, something I was hoping the utility ExternalProject_Add would provide.
The only reason I seem to have noticed is that I'm developing a package that is based on very current packages, on CentOS 6 where the installed versions of libraries are typically older. I was getting a large amount of undefined symbols, etc.
Case Study: Building libtiff using ExternalProject_add, with the options of lzma, jpeg, and zlib = TRUE, passing in the CMAKE_ARG -DCMAKE_PREFIX_PATH=/path/to/cmake/prefix. Assuming liblzma.so, libjpeg.so, and libz.so are in /home/test/lib, and their includes are in /home/test/include, the following always
In a CMakeList.txt in $PROJECT_ROOT/vendors. This is included in the top level CMakeLists.txt
ExternalProject_Add(tiff
URL ${CMAKE_CURRENT_SOURCE_DIR}/libtiff-4.0.9.tar.xz
URL_MD5 56bb233ebdfdb81b0f2464cf54de9d80 #this will not match, it's been untarred and recompressed
PREFIX ${CMAKE_CURRENT_BINARY_DIR}/libtiff
INSTALL_DIR ${PROJECT_ROOT} #/home/test
CMAKE_ARGS -DCMAKE_PREFIX_PATH=<INSTALL_DIR>
-DCMAKE_INSTALL_PREFIX=<INSTALL_DIR>
-DCMAKE_INSTALL_LIBDIR=<INSTALL_DIR>/lib
-DBUILD_SHARED_LIBS=ON
-Dzlib=ON
-Dlzma=ON
-Djpeg=ON
-Dold-jpeg=OFF
-Dcxx=ON
-Djbig=OFF
-Djpeg12=OFF
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
LOG_CONFIGURE 1
LOG_BUILD 1
LOG_INSTALL 1
)
This configures with the following (relevant section) below. All seems well, the libraries were found with the correct versions.
-- Found ZLIB: /home/test/lib/libz.so (found version "1.2.5.3")
-- Found JPEG: /home/test/lib/libjpeg.so (found version "90")
-- Looking for jbg_newlen
-- Looking for jbg_newlen - not found
-- Looking for lzma_auto_decoder in /home/test/lib/liblzma.so
-- Looking for lzma_auto_decoder in /home/test/lib/liblzma.so - found
-- Looking for lzma_easy_encoder in /home/test/lib/liblzma.so
-- Looking for lzma_easy_encoder in /home/test/lib/liblzma.so - found
-- Looking for lzma_lzma_preset in /home/test/lib/liblzma.so
-- Looking for lzma_lzma_preset in /home/test/lib/liblzma.so - found
-- Found LibLZMA: /home/test/include (found version "5.2.4").
Running make VERBOSE=1 tiff
and checking the tiff-build-out.log shows examples where executables are linked against -ljpeg
, -lz
, and -llzma
, instead of the full path that should have been returned by find_package(ZLIB) = /home/test/lib/libz.so
, find_package(LZMA) = /home/test/lib/liblzma.so
, and find_package(JPEG) = /home/test/lib/libjpeg.so
.
[ 72%] Linking C executable tiffdither
cd /dev/shm/ObscurantModeling/build/ThirdPartyLibraries/libtiff/src/tiff-build/tools && /home/bentleykw/bin/cmake -E cmake_link_script CMakeFiles/tiffdither.dir/link.txt --verbose=1
/opt/rh/devtoolset-7/root/usr/bin/gcc -Wall -Winline -W -Wformat-security -Wpointer-arith -Wdisabled-optimization -Wno-unknown-pragmas -Wdeclaration-after-statement -fstrict-aliasing -rdynamic CMakeFiles/tiffdither.dir/tiffdither.c.o -o tiffdither -Wl,-rpath,/dev/shm/ObscurantModeling/build/ThirdPartyLibraries/libtiff/src/tiff-build/libtiff: ../libtiff/libtiff.so.5.3.0 ../port/libport.a -lm -lz -ljpeg -llzma
Afterwards, ldd on the tiffdither binary shows
[tester@linuxpc build]$ ldd /home/test/bin/tiffdither
linux-vdso.so.1 => (0x00007fff1c1d0000)
libtiff.so.5 => /home/test/lib/libtiff.so.5 (0x00007f3bbf627000)
libm.so.6 => /lib64/libm.so.6 (0x00000032be000000)
libz.so.1 => /home/test/lib/libz.so.1 (0x00007f3bbf3e5000)
libjpeg.so.62 => /usr/lib64/libjpeg.so.62 (0x00000032d0600000) <- Here
liblzma.so.0 => /usr/lib64/liblzma.so.0 (0x00000032c9c00000) <- Here
libc.so.6 => /lib64/libc.so.6 (0x00000032bd800000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00000032bdc00000)
/lib64/ld-linux-x86-64.so.2 (0x000055e7ef52c000)
The bold links show that libz was linked correctly, but not jpeg or lzma. This is still in the terminal that built the project, so I don't think it's a library path issue. All environment variables are set as they were during the build.
Is this something that can be avoided? Is it just poor authoring of CMake build systems that should rely on imported targets more often? Any advice would be appreciated.
Thanks,
Kyle B