Help with CMake, Intel MPI, gfortran, 'use mpi'...and symlinks?
A system I work on (for various reasons) will only support at present Intel MPI as the MPI stack. We still want to support GCC compilers in our MPI Fortran project and, for at least GCC 8, Intel MPI will support it as well for the mpif.h
and use mpi
cases. For example this file:
program test
use mpi
implicit none
integer :: ierror
call mpi_init(ierror)
call mpi_finalize(ierror)
end program test
compiles just fine with GCC 8.3.0 and Intel MPI 19.0.2:
$ mpifc test_mpi_mod.f90
$ echo $?
0
But, now I build a fun little CMake project with:
cmake_minimum_required(VERSION 3.14)
project(test VERSION 1.0.0 LANGUAGES Fortran)
find_package(MPI REQUIRED)
message(STATUS "MPI_Fortran_MODULE_DIR: ${MPI_Fortran_MODULE_DIR}")
message (STATUS "MPI_Fortran_HAVE_F77_HEADER: ${MPI_Fortran_HAVE_F77_HEADER}")
if (MPI_Fortran_HAVE_F77_HEADER)
message(STATUS "Building test_mpi_h")
add_executable(test_mpi_h.exe test_mpi_h.f90)
target_link_libraries(test_mpi_h.exe PRIVATE MPI::MPI_Fortran)
endif ()
message (STATUS "MPI_Fortran_HAVE_F90_MODULE: ${MPI_Fortran_HAVE_F90_MODULE}")
if (MPI_Fortran_HAVE_F90_MODULE)
message(STATUS "Building test_mpi_mod")
add_executable(test_mpi_mod.exe test_mpi_mod.f90)
target_link_libraries(test_mpi_mod.exe PRIVATE MPI::MPI_Fortran)
endif ()
message (STATUS "MPI_Fortran_HAVE_F08_MODULE: ${MPI_Fortran_HAVE_F08_MODULE}")
if (MPI_Fortran_HAVE_F08_MODULE)
message(STATUS "Building test_mpi_f08")
add_executable(test_mpi_f08.exe test_mpi_f08.f90)
target_link_libraries(test_mpi_f08.exe PRIVATE MPI::MPI_Fortran)
endif ()
Where the three dummy source files use mpif.h
, use mpi
, and use mpi_f08
with MPI_Init
and MPI_Finalize
as you'd expect.
Now, Intel Fortran 19.0.2 and Intel MPI 19.0.2 output of cmake ..
-- Found MPI_Fortran: /gpfsm/dulocal/sles12/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/lib/libmpifort.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- MPI_Fortran_MODULE_DIR: /gpfsm/dulocal/sles12/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include
-- MPI_Fortran_HAVE_F77_HEADER: TRUE
-- Building test_mpi_h
-- MPI_Fortran_HAVE_F90_MODULE: TRUE
-- Building test_mpi_mod
-- MPI_Fortran_HAVE_F08_MODULE: TRUE
-- Building test_mpi_f08
Okay, worked as expected.
Now let's try GCC 8.3.0 and Intel MPI 19.0.2 with cmake ..
-- Found MPI_Fortran: /gpfsm/dulocal/sles12/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/lib/libmpifort.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- MPI_Fortran_MODULE_DIR: /gpfsm/dulocal/sles12/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include/gfortran/8.2.0
-- MPI_Fortran_HAVE_F77_HEADER: TRUE
-- Building test_mpi_h
-- MPI_Fortran_HAVE_F90_MODULE: FALSE
-- MPI_Fortran_HAVE_F08_MODULE: FALSE
The mpi_f08
is expected as Intel MPI doesn't support that with gfortran. But it didn't find the mpi.mod
even though if I ls
that MPI_Fortran_MODULE_DIR
:
$ ls /gpfsm/dulocal/sles12/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include/gfortran/8.2.0
mpi_base.mod mpi_constants.mod mpi.mod mpi_sizeofs.mod
Huh. Weird. Now, I know on this system that /gpfsm/dulocal/sles12
has a symlink at /usr/local
:
$ ls /usr/local/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include/gfortran/8.2.0
mpi_base.mod mpi_constants.mod mpi.mod mpi_sizeofs.mod
and even though you'd expect a physical directory path to be better than a logical/NFS symlink one, if I pass in that directory to cmake .. -DMPI_Fortran_MODULE_DIR=/usr/local/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include/gfortran/8.2.0
then:
-- Found MPI_Fortran: /gpfsm/dulocal/sles12/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/lib/libmpifort.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- MPI_Fortran_MODULE_DIR: /usr/local/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include/gfortran/8.2.0
-- MPI_Fortran_HAVE_F77_HEADER: TRUE
-- Building test_mpi_h
-- MPI_Fortran_HAVE_F90_MODULE: TRUE
-- Building test_mpi_mod
-- MPI_Fortran_HAVE_F08_MODULE: FALSE
And it sees mpi.mod
!
But (and this is the dot in Jeremy Bearimy to me), I run make VERBOSE=1
with the above cmake:
[ 25%] Building Fortran object CMakeFiles/test_mpi_mod.exe.dir/test_mpi_mod.f90.o
/usr/local/other/gcc/8.3.0/bin/gfortran -I/gpfsm/dulocal/sles12/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include/gfortran/8.2.0 -I/gpfsm/dulocal/sles12/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include -I/usr/local/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include/gfortran/8.2.0 -c /home/mathomp4/cmakempitest/test_mpi_mod.f90 -o CMakeFiles/test_mpi_mod.exe.dir/test_mpi_mod.f90.o
it has the gfortran/8.2.0
directory twice, the autodetected one and the one I forced in. But if I remove the second -I (so keeping only the autodetected gpfsm
path) and run by hand on the command line:
$ /usr/local/other/gcc/8.3.0/bin/gfortran -I/gpfsm/dulocal/sles12/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include/gfortran/8.2.0 -I/gpfsm/dulocal/sles12/intel/2019/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include -c /home/mathomp4/cmakempitest/test_mpi_mod.f90
$ echo $?
0
So my questions are:
- Any ideas on what is happening and why I'd need to do that? If anything I'd expect the physical (in the
pwd -P
) directory path to be more stable than the logical/symlink one, and yet... - In the sense of "okay I have a workaround" in re running CMake manually, did I do this right? Is
MPI_Fortran_MODULE_DIR
the right thing to set at the command line? - Any hints on what could be a good way to automate this in CMake land? Trying to figure out logical and physical paths?