Update mpi.cmake to utilize UCX if requested
The goal is to take advantage of infiniband devices utilizing the channel for ucx. I have added an argument that will reset the "default device channel" if it is enabled. I have also added another argument that will give a user an opportunity to point towards an external version of UCX. Recently, I myself ran into a few issues, on an HPC, that needed a newer version in order to resolve them. However, in order to utilize other arguments, a user must enable ucx. Otherwise, the build will be the same as it is now.
Example 1:
cmake \
-DENABLE_mpi:BOOL=ON \
-DUSE_SYSTEM_mpi:BOOL=OFF \
-DMPI_ENABLE_UCX:BOOL=ON \
-DMPI_ADDITIONAL_CONFIG_ARGS:STRING="--enable-debuginfo;--enable-cxx" \
-DCMAKE_INSTALL_PREFIX=$prefix \
../src && make
Example 2:
cmake \
-DENABLE_mpi:BOOL=ON \
-DUSE_SYSTEM_mpi:BOOL=OFF \
-DMPI_ENABLE_UCX:BOOL=ON \
-DMPI_ADDITIONAL_CONFIG_ARGS:STRING="--with-ucx=[dir];--enable-debuginfo" \
-DCMAKE_INSTALL_PREFIX=$prefix \
../src && make