FindMPI: manually specifying libs doesn't work
First of all, the specific problem I have, getting cmake to work on spock while using hipcc as C++ compiler is better handled using !6264 (merged). But manually specifying MPI includes/libs is still something that should work, according to the docs:
If all of these parameters were not sufficient to find the right MPI implementation, a user may disable the entire autodetection process by specifying both a list of libraries in MPI__LIBRARIES and a list of include directories in MPI__ADDITIONAL_INCLUDE_DIRS. Any other variable may be set in addition to these two. The module will then validate the MPI settings and store the settings in the cache.
If someone wants to reproduce this exactly on spock, here are the modules I have loaded:
Currently Loaded Modules:
1) gcc/10.3.0 5) PrgEnv-gnu/8.1.0 9) libfabric/1.11.0.4.75 13) fftw/3.3.9
2) craype/2.7.8 6) DefApps/default 10) craype-network-ofi 14) netlib-scalapack/2.1.0
3) cray-dsmml/0.1.5 7) craype-x86-rome 11) cray-mpich/8.1.8 15) cmake/3.20.2
4) cray-libsci/21.06.1.1 8) craype-accel-amd-gfx908 12) rocm/4.3.0
But this should be reproducible elsewhere without too much trouble. Here's a reproducer CMakeLists.txt
:
cmake_minimum_required(VERSION 3.20)
project(test LANGUAGES CXX)
find_package(MPI REQUIRED)
On spock, I call cmake like this:
[kaig1@login1 build]$ cmake -S .. \
-DCMAKE_CXX_COMPILER=`which hipcc` \
-DMPI_CXX_LIBRARIES="${MPICH_DIR}/ofi/gnu/9.1/lib/libmpi_gnu_91.so;$MPICH_DIR/gtl/lib/libmpi_gtl_hsa.so" \
-DMPI_CXX_ADDITIONAL_INCLUDE_DIRS="$MPICH_DIR/ofi/gnu/9.1/include"
-- The CXX compiler identification is Clang 13.0.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/rocm-4.3.0/bin/hipcc - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find MPI_CXX (missing: MPI_CXX_LIB_NAMES) (found version "3.1")
CMake Error at /autofs/nccs-svm1_sw/spock/spack-envs/base/opt/cray-sles15-zen2/gcc-10.3.0/cmake-3.20.2-vcqct2noa2xiirar44aau7r3tw673su2/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find MPI (missing: MPI_CXX_FOUND) (found version "3.1")
What happens is that the specified libraries are found, and then they are put into MPI_CXX_EXTRA_LIB_NAMES
, but they never make it into MPI_CXX_LIB_NAMES
, hence the failure.
I don't think I could fully decipher FindMPI.cmake
. There is one place which will take the extra lib names and put them into MPI_*_LIB_NAMES
, but it's in the case of MPI_PINNED_COMPILER
being true. I can force that branch by adding -DMPI_CXX_COMPILER=$(which hipcc)
, but I don't think I should have to do that, and in any case this still fails -- the lib names get set correctly, but when it checks whether the MPI_CXX compile works, it fails because the header / library path hasn't been added during the check.
In any case, for all I can tell this looks to be a more generic problem where manually specifying headers and libraries doesn't work correctly.