10.6 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120
# Common Superbuild Infrastructure

This project started as a refactor of a number of projects which had started
using the ParaView superbuild, but had diverged over time. Features were
added, patches trickled back and forth, but the gap needed to be closed. This
repository represents the common ground between all of them and aims to do so
in a flexible way so that such gaps do not grow too large again.

# Defining projects

Projects are defined under the `projects/` hierarchy. Under this hierarchy,
files with the name `$project.cmake` are created which define how to build the
project. Other files which have precedence include `$project.common.cmake`,
`$project.functions.cmake`, and `$project.system.cmake`.

  * `$project.common.cmake`: for projects which differ greatly based on the
    platform, this is intended to be included by platform-specific files and
    tweaked using variables.
  * `$project.functions.cmake`: some projects might offer functions to
    simplify things such as installation or testing. These functions should
    be defined in a separate file so that they may be available during
    packaging and testing as well.
  * `$project.system.cmake`: projects may represent software packages which
    are available as a system-wide copy; this file is used to find the system
    copy and import it into the superbuild infrastructure.

Different platforms can shadow these files by using the following

  * Windows: `win32/`
  * OS X: `apple/` and `apple-unix/`
  * Linux: `unix/` and `apple-unix/`

Usually these will just be the `$project.cmake` files, patches, and helper
scripts, but it is not impossible that shadowing for the other files would be

## Related files

Some projects might need support files such as patches or small scripts to
complete some parts of the build step. For these files, the directories
`patches/` and `scripts/` should be used. The files should start with the name
of the project they are used for so that it is obvious why they exist.
Additionally, they should be underneath the directory for the `$project.cmake`
file which uses them. This keeps, e.g., Windows-specific patches underneath
the Windows platform directory.

Scripts should, preferably, be written in CMake code, but as a fallback,
Python should be preferred since it is generally available. Shell and Batch
scripts should be avoided.

## Guidelines

Within a `$project.cmake` and related files, some guidelines should be

  - clear variables before use: the two-pass setup used by the superbuild may
    cause variables to be non-empty on the second pass;
  - prefix all variables and functions with the project name: variables such
    as `extra_flags` and such might conflict with other projects;
  - comment complicated logic: projects might need non-trivial logic to work
    around weird bugs or other build failures; by documenting why this logic
    exists, the idea is that time will not be needed to rediscover the reason
    it exists.

## Examples

*TODO*: Add some examples of projects.

# Using the superbuild infrastructure

To create a new superbuild, a repository should be created which contains the
superbuild infrastructure as a submodule. This allows updating to new versions
of the common infrastructure easily and keeps the version which is being used
easy to track.

There is only one required part (`superbuild_find_projects`), but other things
will generally be necessary for a new project, particularly
`superbuild_project_roots` and `superbuild_version_files`. See below for more
specific information on these. After defining the functions and setting up the
variables, all that is required is to add the infrastructure's directory using
`add_subdirectory`. It uses the functions and the variables values to do all
of the required work. This setup is done so that the infrastructure can do
work in between all of the functions without application superbuild needing to
call the right functions in between the different steps.

*Note*: For the purposes of the documentation, the new repository will be referred to
as pertaining to an application to keep the use of the term "project" less
ambiguous. By no means is the superbuild limited to building end-user
applications (though it is the primary use case).

## `superbuild_find_projects` (function)

The core bit which is required by the superbuild infrastructure to work is a
function called `superbuild_find_projects`. This function is used to populate
a variable (passed in as the argument) with the list of projects that are used
for the application. All projects which are required for the application
should be added to the list. As an example:

function (superbuild_find_projects var)
        application dep1 dep2)

    if (WIN32)
        list(APPEND projects
    endif ()

    set("${var}" "${projects}" PARENT_SCOPE)
endfunction ()

Here, the application depends on two other projects, `dep` and `dep2`.
Additionally, on Windows, `depforwin32` is also a dependency.

The superbuild infrastructure will only consider these projects to exist. Any
other projects are completely ignored.

## `superbuild_project_roots` (variable)

122 123 124 125
This variable is a list of directory paths to treat as roots to look for
`$project.cmake` (and related) files. The relevant platforms will
automatically be searched for within each root. In addition, the common
infrastructure will put its project root at the end of this list.

The typical directory name for these roots is `projects/`. As an example:

129 130 131 132
list(APPEND superbuild_project_roots

## `superbuild_version_files` (variable)

136 137 138
This variable is a list of files to include to declare version information for
the projects which will be built. The usual file name for these files is
`versions.cmake`. The following functions will be available in the file:

140 141 142
  - `superbuild_set_revision`
  - `superbuild_set_customizable_revision`
  - `superbuild_set_external_source`

144 145
See the [SuperbuildRevisionMacros][revision-macros] file for documentation on
these functions.

147 148 149
The common infrastructure will place its file at the end and it defines
version information for the projects it ships. Applications may override these
versions by defining its own in a file in this list.

## `superbuild_setup_variables` (function)

153 154 155
This function (though usually a macro), if defined, is called after revisions
are defined, but before any projects are included. This is where project-wide
variables should be set so that they are available everywhere in the project.

157 158 159 160
This is usually where the `superbuild_set_version_variables` function is
called (see the [SuperbuildVersionMacros][version-macros] file for its
documentation) so that the application's version information is available at
all times.

## `superbuild_sanity_check` (function)

164 165 166 167 168 169
This function, if defined, is called after setting up the entire build. At
this stage, variables of the form `${project}_enabled` are available which
indicate whether a project will be built or not. This function is the place to
check whether project settings conflict with each other or whether an
unsupported configuration has been chosen (e.g., not building the application,
or choosing incompatible versions of Python).

## Post-build tasks

173 174
The infrastructure supports running post-build tasks such as packaging and
testing. This uses CPack and CTest.

176 177 178 179 180 181 182
The preferred method to packaging is to set up an unrelated project which is
basically nothing except for installation logic. This is because CPack likes
to rerun the installation of the top-level project before it runs which can
retrigger builds of the main project. If projects are chasing branches in git
and the branch has updated since the build and packaging, all of a sudden
you're packaging different code than you had built. As such, the old,
deprecated process is not documented here.

### `superbuild_add_packaging` (function)

186 187 188 189 190
This function, if defined, should call the `superbuild_add_extra_package_test`
function (see [SuperbuildPackageMacros][package-macros] for documentation) to
create a test which will create a package using a specified generator. Tests
are described by `$project.bundle.cmake` files in the project root

192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250
The bundle files will have the following variables available:

  - `superbuild_install_location`
  - `superbuild_source_directory`
  - `superbuild_is_64bit`
  - `${project}_enabled`
  - `USE_SYSTEM_${project}`

In addition, variables may be exported to the bundle project by setting the
`superbuild_export_variables` variable to a list of variable names to add.

See the [SuperbuildInstallMacros][install-macros] file for documentation on
the variables available in `$project.bundle.cmake` files.

### `superbuild_add_tests` (function)

This function, if defined, will enable testing for the superbuild itself.
Before testing is enabled, the `superbuild_setup_tests` hook is called (if
set) so that any required variables may be set.

At the top-level (which should be fixed at some point; the function should do
this), the application may set the `superbuild_ctest_custom_files` variable to
a list of paths to `CTestCustom.cmake` files to be included during testing.
Within these files, convenience functions to ignore warnings and errors from
projects are provided:

  - `ignore_project_warnings(project)`
  - `ignore_project_errors(project)`

This may be used to silence warnings or errors from projects which cause noise
on the dashboard.

In addition, the `superbuild_test_projects` global property may be set to a
list of projects which should additionally be tested during a test of the
superbuild. This will cause the project's tests to be available and run as
part of a single top-level run of CTest. Cache variables named
`TEST_${project}` will be added so that the user can disable this behavior. If
the project is either disabled or in a developer mode, the tests are similarly

## Advanced customization

### `superbuild_install_location` (variable)

This variable is a path where all of the projects will install themselves. If
it is not set, the default value is `${CMAKE_BINARY_DIR}/install`.

### Flags

The common infrastructure will bring in flags from both the user's settings
and the application. User settings are grabbed from the `CMAKE_C_FLAGS`,
`CMAKE_CXX_FLAGS`, and `CMAKE_SHARED_LINKER_FLAGS` variables. Application
flags use `superbuild_ld_flags`, `superbuild_cpp_flags`,
`superbuild_cxx_flags`, and `superbuild_c_flags`.

[install-macros]: cmake/SuperbuildInstallMacros.cmake
[package-macros]: cmake/SuperbuildPackageMacros.cmake
[revision-macros]: cmake/SuperbuildRevisionMacros.cmake
[version-macros]: cmake/SuperbuildVersionMacros.cmake