Particle advection build failing on Perlmutter (CUDA + MPI issue)
The particle advection filter fails to build on Perlmutter (CUDA + MPI), but the issue may be a broader CUDA + MPI issue (do we have any test builds with CUDA and MPI)?
[ 49%] Building CUDA object vtkm/filter/CMakeFiles/vtkm_filter_extra.dir/particleadvection/Messenger.cxx.o
cd /global/homes/g/ghweber/devel/vtk-m/build-perlmutter/vtkm/filter && /opt/nvidia/hpc_sdk/Linux_x86_64/21.11/cuda/11.5/bin/nvcc -forward-unknown-to-host-compiler -DVTKMDIY_DEBUG -DVTKMDIY_MPI_AS_LIB -DVTKMDIY_NO_THREADS -Dvtkm_filter_extra_EXPORTS -I/global/homes/g/ghweber/devel/vtk-m -I/global/homes/g/ghweber/devel/vtk-m/build-perlmutter/include -I/global/homes/g/ghweber/devel/vtk-m/vtkm/thirdparty/optionparser -I/global/homes/g/ghweber/devel/vtk-m/vtkm/thirdparty/diy -I/global/homes/g/ghweber/devel/vtk-m/vtkm/thirdparty/lcl/vtkmlcl -isystem=/global/homes/g/ghweber/devel/vtk-m/vtkm/thirdparty/diy/vtkmdiy/include -isystem=/global/homes/g/ghweber/devel/vtk-m/build-perlmutter/vtkm/thirdparty/diy/vtkmdiy/include/vtkmdiy/mpi --generate-code=arch=compute_80,code=sm_80 -g -Xcompiler=-fPIC -Xcompiler=-fvisibility=hidden -Xcompiler=-Wall,-Wcast-align,-Wpointer-arith,-Wformat,-Wformat-security,-Wshadow,-fno-common,-Wunused,-Wno-unknown-pragmas,-Wno-unused-local-typedefs,-Wno-unused-function,-Wchar-subscripts,-Wfloat-conversion,-Wodr -Xcudafe=--display_error_number -Xcompiler=-ffunction-sections --expt-relaxed-constexpr -std=c++14 -MD -MT vtkm/filter/CMakeFiles/vtkm_filter_extra.dir/particleadvection/Messenger.cxx.o -MF CMakeFiles/vtkm_filter_extra.dir/particleadvection/Messenger.cxx.o.d -x cu -c /global/homes/g/ghweber/devel/vtk-m/vtkm/filter/particleadvection/Messenger.cxx -o CMakeFiles/vtkm_filter_extra.dir/particleadvection/Messenger.cxx.o
In file included from /global/homes/g/ghweber/devel/vtk-m/vtkm/filter/particleadvection/Messenger.cxx:15:
/global/homes/g/ghweber/devel/vtk-m/vtkm/filter/particleadvection/Messenger.h:26:10: fatal error: mpi.h: No such file or directory
26 | #include <mpi.h>
| ^~~~~~~
compilation terminated.
make[2]: *** [vtkm/filter/CMakeFiles/vtkm_filter_extra.dir/build.make:76: vtkm/filter/CMakeFiles/vtkm_filter_extra.dir/particleadvection/Messenger.cxx.o] Error 1
make[2]: Leaving directory '/global/u2/g/ghweber/devel/vtk-m/build-perlmutter'
make[1]: *** [CMakeFiles/Makefile2:4037: vtkm/filter/CMakeFiles/vtkm_filter_extra.dir/all] Error 2
make[1]: Leaving directory '/global/u2/g/ghweber/devel/vtk-m/build-perlmutter'
make: *** [Makefile:166: all] Error 2
I can continue the build if I manually add a -I$MPICH_DIR/include
to the end of the compile line. It seems the MPI include options do not get added properly for compiling device/CUDA sources.