NVHPC & HPE: Unified Heterogeneous CUDA Compiles

Hi,

I learned today that on NERSC's Perlmutter, Nvidia's support for NVHPC is a bit wild on compiler pairing.

In particular, using the former PGI compilers nvc++ as host compiler for nvcc is only supported via nvcc -ccbin=nvc++ but not via nvcc -ccbin=/absolute/path/to/nvc++.

Unfortunately, both the -DCMAKE_CUDA_HOST_COMPILER=nvc++ and the CUDAHOSTCXX environment hint get expanded to absolute paths at the moment.

I am unsure if we need to work-around this in CMake, if that's a NERSC issue or an Nvidia issue to report. It's definitely an unusual/very specific choice from Nvidia for this flag.

Update

So it looks to me like PrgEnv-nvidia on HPE/Cray might depend on the already announced NVHPC unified heterogeneous compiler support once it is complete.

As I see so far, it looks like unified heterogeneous compiles of CUDA targets using nvc++ or HPE/Cray CC are not yet implemented in CMake (?).

The combination of nvcc -ccbin=nvc++ itself might receive less attention than the announced support for unified heterogeneous builds directly in Nvidia nvc++/HPE CC.