|
|
**This page is work in progress \!**
|
|
|
|
|
|
# How to build software for Cray Xt3 / Catamount
|
|
|
|
|
|
Cray Xt3 consists of two different types of nodes, the front end node,
|
|
|
which are more or less regular AMD64 Linux machines, and the actual
|
|
|
compute nodes, which have the same processors, but run the Catamount as
|
|
|
operating system. Building software for the front end nodes is not
|
|
|
different than building software on any other Linux system. But to build
|
|
|
software for the actual compute nodes, you need to cross compile. For
|
|
|
C/C++ there are typically two toolchains you can use for the compute
|
|
|
nodes:
|
|
|
|
|
|
- the GNU gcc toolchain
|
|
|
- the Portland Group PGI toolchain
|
|
|
|
|
|
In order to use them with CMake you have to write a toolchain file.
|
|
|
|
|
|
There are wrapper scripts provided to call the actual compilers. There
|
|
|
is a cc script for the C compiler, and a CC script for the C++ compiler.
|
|
|
Depending on the current "environment" they call either the GNU compiler
|
|
|
or the PGI compiler. By default the PGI compiler is used. To switch to
|
|
|
gcc, use:
|
|
|
|
|
|
$ module switch PrgEnv-pgi PrgEnv-gnu
|
|
|
|
|
|
To switch back, exchange the two last parameters. These scripts add a
|
|
|
several include directories, compile flags, link directories and
|
|
|
libraries to the compile and link calls.
|
|
|
|
|
|
CMake currently supports these wrapper scripts. There is a problem here.
|
|
|
Since the same compiler executable ("cc") can be two different compilers
|
|
|
depending on the currently loaded environment, it can happen that some
|
|
|
part of a project is compiled with PGI, then the user switches to gcc,
|
|
|
and when he continues the build, gcc builds the rest of the files.
|
|
|
|
|
|
C++ libraries between PGI and g++ seem to be not ABI compatible.
|
|
|
Additionally it seems -lm may not be mixed, i.e. if some library is
|
|
|
compiled with PGI and uses the math library and some other part is
|
|
|
compiled with gcc and uses the math library there may be linker
|
|
|
problems. So it may be the better choice to decide for one of the two
|
|
|
toolchains and use this for everything.
|
|
|
|
|
|
## Writing the CMake toolchain file
|
|
|
|
|
|
For CMake to be able to crosscompile software, it requires you to write
|
|
|
a toolchain file, which tells CMake some information about the
|
|
|
toolchain.
|
|
|
|
|
|
The toolchain on for Cray Xt3/Catamount could look like like the
|
|
|
following:
|
|
|
|
|
|
# the name of the target operating system
|
|
|
SET(CMAKE_SYSTEM_NAME Catamount)
|
|
|
|
|
|
# set the compiler
|
|
|
set(CMAKE_C_COMPILER cc -target=catamount)
|
|
|
set(CMAKE_CXX_COMPILER CC -target=catamount)
|
|
|
|
|
|
# set the search path for the environment coming with the compiler
|
|
|
# and a directory where you can install your own compiled software
|
|
|
set(CMAKE_FIND_ROOT_PATH
|
|
|
/opt/xt-pe/default
|
|
|
/opt/xt-mpt/default/mpich2-64/GP2
|
|
|
/home/alex/cray-install
|
|
|
)
|
|
|
|
|
|
# adjust the default behaviour of the FIND_XXX() commands:
|
|
|
# search headers and libraries in the target environment, search
|
|
|
# programs in the host environment
|
|
|
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
|
|
|
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
|
|
|
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
|
|
|
|
|
|
Without more special settings this will work both for PGI and gcc. Save
|
|
|
this file as Toolchain-Catamount.cmake to some location where you will
|
|
|
put all your toolchain files, e.g. $HOME/Toolchains/. As you can see
|
|
|
CMAKE_FIND_ROOT_PATH is set to three directories coming with the
|
|
|
toolchain, which contain the headers and libraries installed with the
|
|
|
toolchain, and /home/alex/cray-install/. This last directory is intended
|
|
|
to hold other libraries you will compile for the compute nodes, they
|
|
|
should be installed under this install prefix. This way the FIND_XXX()
|
|
|
commands in CMake will find both the headers and libraries coming with
|
|
|
the toolchain as well as additional libraries you have built for this
|
|
|
platform.
|
|
|
|
|
|
## Building the software for Catamount with the GNU toolchain
|
|
|
|
|
|
Let's say you have the classical hello world software with a CMake based
|
|
|
buildsystem and want to build this for now for your super computer.
|
|
|
main.c:
|
|
|
|
|
|
#include <stdio.h>
|
|
|
|
|
|
int main()
|
|
|
{
|
|
|
printf("Hello world\n");
|
|
|
return 0;
|
|
|
}
|
|
|
|
|
|
CMakeLists.txt:
|
|
|
|
|
|
ADD_EXECUTABLE(hello main.c)
|
|
|
|
|
|
Then run CMake on it to generate the buildfiles, the important point is
|
|
|
that you tell it to use the toochain file you just wrote:
|
|
|
|
|
|
~/src/helloworld/ $ module switch PrgEnv-pgi PrgEnv-gnu
|
|
|
~/src/helloworld/ $ mkdir build-gcc
|
|
|
~/src/helloworld/ $ cd build-gcc
|
|
|
~/src/helloworld/build-gcc/ $ cmake -DCMAKE_TOOLCHAIN_FILE=~/Toolchains/Toolchain-Catamount.cmake -DCMAKE_INSTALL_PREFIX=/home/alex/cray-install ..
|
|
|
-- The C compiler identification is GNU
|
|
|
-- The CXX compiler identification is GNU
|
|
|
...
|
|
|
-- Configuring done
|
|
|
-- Generating done
|
|
|
-- Build files have been written to: /home/alex/src/helloworld/build-gcc
|
|
|
~/src/helloworld/build-gcc/ $ make
|
|
|
Scanning dependencies of target hello
|
|
|
[100%] Building C object CMakeFiles/hello.dir/main.o
|
|
|
Linking C executable hello
|
|
|
[100%] Built target hello
|
|
|
|
|
|
So that's all. It actually doesn't matter whether it's just a "hello
|
|
|
world" or some complex piece of software, the only difference is the
|
|
|
usage of the toolchain file. If the software has all required configure
|
|
|
checks, it should just build also with this toolchain.
|
|
|
|
|
|
## Unsupported features of Catamount
|
|
|
|
|
|
Among others Catamount doesn't support the following features:
|
|
|
|
|
|
- shared libraries
|
|
|
- multithreading
|
|
|
- IP server sockets
|
|
|
- IP client sockets
|
|
|
- getpw\*() and other functions which might use nss_switch
|
|
|
|
|
|
Some functions like e.g. times() are present in libc, but the linker
|
|
|
warns:
|
|
|
|
|
|
: warning: warning: times is not implemented and will always fail
|
|
|
|
|
|
This is the case for several functions, not only times(). So while the
|
|
|
linker can resolve the symbols, the functions will fail at runtime. It
|
|
|
may be a good idea to use `-Wl,--fatal-warnings`, so that the linker
|
|
|
will interpret these warnings as errors and you as developer will
|
|
|
actually notice if something breaks.
|
|
|
|
|
|
----
|
|
|
This page was initially populated by conversion from its [original location](https://public.kitware.com/Wiki/CmakeCrayXt3) in another wiki. |