|
|
# How to build software for IBM/BlueGene
|
|
|
|
|
|
BlueGene consists of two different types of nodes, the service node,
|
|
|
which are more or less regular PowerPC Linux machines, and the actual
|
|
|
compute nodes, which have the same processors, but run the "Compute Node
|
|
|
Kernel" as operating system. Building software for the service nodes is
|
|
|
not different than building software on any other Linux system. But to
|
|
|
build software for the actual compute nodes, you need to cross compile.
|
|
|
For C/C++ there are typically two toolchains you can use for the compute
|
|
|
nodes:
|
|
|
|
|
|
- the GNU gcc toolchain (powerpc-bgl-blrts-gnu-gcc)
|
|
|
- the IBM XL toolchain (blrts_xlc)
|
|
|
|
|
|
In order to use them with CMake you have to write a toolchain file, more
|
|
|
on that below.
|
|
|
|
|
|
## Writing the CMake toolchain file for the GNU toolchain
|
|
|
|
|
|
For CMake to be able to crosscompile software, it requires you to write
|
|
|
a toolchain file, which tells CMake some information about the
|
|
|
toolchain.
|
|
|
|
|
|
For the GNU toolchain on BlueGene it could look like like the following:
|
|
|
|
|
|
# the name of the target operating system
|
|
|
SET(CMAKE_SYSTEM_NAME BlueGeneL)
|
|
|
|
|
|
# set the compiler
|
|
|
set(CMAKE_C_COMPILER /bgl/BlueLight/ppcfloor/blrts-gnu/bin/powerpc-bgl-blrts-gnu-gcc )
|
|
|
set(CMAKE_CXX_COMPILER /bgl/BlueLight/ppcfloor/blrts-gnu/bin/powerpc-bgl-blrts-gnu-g++ )
|
|
|
|
|
|
# set the search path for the environment coming with the compiler
|
|
|
# and a directory where you can install your own compiled software
|
|
|
set(CMAKE_FIND_ROOT_PATH
|
|
|
/bgl/BlueLight/ppcfloor/
|
|
|
/bgl/BlueLight/V1R3M2_140_2007-070424/ppc/blrts-gnu/powerpc-bgl-blrts-gnu/
|
|
|
/bgl/BlueLight/V1R3M2_140_2007-070424/ppc/bglsys
|
|
|
/home/alex/bgl-install
|
|
|
)
|
|
|
|
|
|
# adjust the default behaviour of the FIND_XXX() commands:
|
|
|
# search headers and libraries in the target environment, search
|
|
|
# programs in the host environment
|
|
|
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
|
|
|
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
|
|
|
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
|
|
|
|
|
|
Save this file as Toolchain-BlueGeneL-gcc.cmake to some location where
|
|
|
you will put all your toolchain files, e.g. $HOME. As you can see
|
|
|
CMAKE_FIND_ROOT_PATH is set to three directories coming with the
|
|
|
toolchain, which contain the headers and libraries installed with the
|
|
|
toolchain, and /home/alex/bgl-install/. This last directory is intended
|
|
|
to hold other libraries you will compile for the compute nodes, they
|
|
|
should be installed under this install prefix. This way the FIND_XXX()
|
|
|
commands in CMake will find both the headers and libraries coming with
|
|
|
the toolchain as well as additional libraries you have built for this
|
|
|
platform.
|
|
|
|
|
|
## Building the software for BlueGene/L with the GNU toolchain
|
|
|
|
|
|
Let's say you have the classical hello world software with a CMake based
|
|
|
buildsystem and want to build this for now for your super computer.
|
|
|
main.c:
|
|
|
|
|
|
#include <stdio.h>
|
|
|
|
|
|
int main()
|
|
|
{
|
|
|
printf("Hello world\n");
|
|
|
return 0;
|
|
|
}
|
|
|
|
|
|
CMakeLists.txt:
|
|
|
|
|
|
ADD_EXECUTABLE(hello main.c)
|
|
|
|
|
|
Then run CMake on it to generate the buildfiles, the important point is
|
|
|
that you tell it to use the toochain file you just wrote:
|
|
|
|
|
|
~/src/helloworld/ $ mkdir build-gcc
|
|
|
~/src/helloworld/ $ cd build-gcc
|
|
|
~/src/helloworld/build-gcc/ $ cmake -DCMAKE_TOOLCHAIN_FILE=~/Toolchain-BlueGeneL-gcc.cmake -DCMAKE_INSTALL_PREFIX=/home/alex/bgl-install ..
|
|
|
-- The C compiler identification is GNU
|
|
|
-- The CXX compiler identification is GNU
|
|
|
...
|
|
|
-- Configuring done
|
|
|
-- Generating done
|
|
|
-- Build files have been written to: /home/alex/src/helloworld/build-gcc
|
|
|
~/src/helloworld/build-gcc/ $ make
|
|
|
Scanning dependencies of target hello
|
|
|
[100%] Building C object CMakeFiles/hello.dir/main.o
|
|
|
Linking C executable hello
|
|
|
[100%] Built target hello
|
|
|
|
|
|
So that's all. It actually doesn't matter whether it's just a "hello
|
|
|
world" or some complex piece of software, the only difference is the
|
|
|
usage of the toolchain file. If the software has all required configure
|
|
|
checks, it should just build also with this toolchain. To run this
|
|
|
program, write an appropriate submit script and run it via llsubmit. If
|
|
|
everything works out fine, you will get output files with one greeting
|
|
|
from each of the processors of the compute node. :-)
|
|
|
|
|
|
CMake can help you a bit with creating this script. Create a file like
|
|
|
the following in the source directory and save it as run_job.ksh.in:
|
|
|
|
|
|
#!/bin/ksh
|
|
|
# @ job_type = BlueGene
|
|
|
# @ executable = /bgl/BlueLight/ppcfloor/bglsys/bin/mpirun
|
|
|
# @ arguments = -cwd ${CMAKE_BINARY_DIR} -exe ${EXE_LOCATION} -mode VN
|
|
|
# @ input = /home/alex/empty_file
|
|
|
# @ output = ${CMAKE_BINARY_DIR}/output.$(jobid).out
|
|
|
# @ error = ${CMAKE_BINARY_DIR}/output.$(jobid).err
|
|
|
# @ initialdir = ${CMAKE_BINARY_DIR}
|
|
|
# @ notify_user = alex.neundorf@kitware.com
|
|
|
# @ notification = complete
|
|
|
# @ wall_clock_limit = 01:05:00
|
|
|
# @ restart = no
|
|
|
# @ coschedule = no
|
|
|
# @ bg_size = 128
|
|
|
# @ queue
|
|
|
|
|
|
Then add the following code to you CMakeLists.txt:
|
|
|
|
|
|
get_target_property(EXE_LOCATION hello LOCATION)
|
|
|
configure_file(run_job.ksh.in ${CMAKE_BINARY_DIR}/run_job.ksh)
|
|
|
|
|
|
When running CMake to create the Makefiles it will replace the
|
|
|
directories and will create a working run_job.ksh in your build
|
|
|
directory.
|
|
|
|
|
|
## Using the IBM XL toolchain for building the software for BlueGene/L
|
|
|
|
|
|
The IBM XL toolchain can also be used with CMake on BlueGene/L. You have
|
|
|
to write a separate toolchain file for it, the main difference to the
|
|
|
one above is that the IBM compilers are used:
|
|
|
|
|
|
# the name of the target operating system
|
|
|
SET(CMAKE_SYSTEM_NAME BlueGeneL)
|
|
|
|
|
|
# set the compiler
|
|
|
set(CMAKE_C_COMPILER /opt/ibmcmp/vac/bg/8.0/bin/blrts_xlc)
|
|
|
set(CMAKE_CXX_COMPILER /opt/ibmcmp/vacpp/bg/8.0/bin/blrts_xlC)
|
|
|
|
|
|
# set the search path for the environment coming with the compiler
|
|
|
# and a directory where you can install your own compiled software
|
|
|
set(CMAKE_FIND_ROOT_PATH
|
|
|
/bgl/BlueLight/ppcfloor/
|
|
|
/bgl/BlueLight/V1R3M2_140_2007-070424/ppc/bglsys
|
|
|
/home/alex/bgl-install
|
|
|
)
|
|
|
|
|
|
# adjust the default behaviour of the FIND_XXX() commands:
|
|
|
# search headers and libraries in the target environment, search
|
|
|
# programs in the host environment
|
|
|
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
|
|
|
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
|
|
|
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
|
|
|
|
|
|
Save this file as Toolchain-BlueGeneL-xlc.cmake in the same directory
|
|
|
where you saved the toolchain file for the GNU toolchain. Then create a
|
|
|
new build directory and run CMake there:
|
|
|
|
|
|
~/src/helloworld/ $ mkdir build-xlc
|
|
|
~/src/helloworld/ $ cd build-xlc
|
|
|
~/src/helloworld/build-xlc/ $ cmake -DCMAKE_TOOLCHAIN_FILE=~/Toolchain-BlueGeneL-xlc.cmake -DCMAKE_INSTALL_PREFIX=/home/alex/bgl-install ..
|
|
|
-- The C compiler identification is VisualAge
|
|
|
-- The CXX compiler identification is VisualAge
|
|
|
...
|
|
|
-- Configuring done
|
|
|
-- Generating done
|
|
|
-- Build files have been written to: /home/alex/src/helloworld/build-xlc
|
|
|
~/src/helloworld/build-xlc/ $ make
|
|
|
Scanning dependencies of target hello
|
|
|
[100%] Building C object CMakeFiles/hello.dir/main.o
|
|
|
Linking C executable hello
|
|
|
[100%] Built target hello
|
|
|
|
|
|
----
|
|
|
This page was initially populated by conversion from its [original location](https://public.kitware.com/Wiki/CmakeBlueGene) in another wiki. |