BuildSection.tex 12 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14
\section{Building ParaView Catalyst}
This section is targeted towards those users or developers responsible for building ParaView
Catalyst. As far as installation is concerned, Catalyst is a subset of the ParaView code base.
Thus, all of the functionality available in Catalyst is also available in ParaView. The difference is
that ParaView will by default have many more dependencies and thus will have a larger
executable size. Catalyst is the flexible and specialized configuration of ParaView that is used to
reduce the executable size by reducing dependencies. For example, if no output images are to
be produced from a Catalyst-instrumented simulation run then all of the ParaView and VTK
code related to rendering and the OpenGL libraries need not be linked in. This can result in
significant memory savings, especially when considering the number of processes utilized when
running a simulation at scale. In one simple example, the executable size was reduced from 75
MB when linking with ParaView to less than 20 MB when linking with Catalyst.
The main steps for configuring Catalyst are:
\begin{enumerate}
15
\item Set up an “edition”
16 17 18 19 20 21 22 23 24
\item Extract the desired code from ParaView source tree into a separate Catalyst source tree
\item Build Catalyst
\end{enumerate}
Most of the work is in the first step which is described below. A Catalyst edition is a
customization of ParaView to support a desired subset of functionality from ParaView and VTK.
There can be many editions of Catalyst and these editions can be combined to create several
customized Catalyst builds. Assuming that the desired editions have already been created, the
second step is automated and is done by invoking the following command from the
\textless ParaView\_source\_dir\textgreater /Catalyst directory:
25
\begin{minted}{python}
Nicolas Vuaille's avatar
Nicolas Vuaille committed
26
python catalyze.py -r <paraview_sources> -i <edition_dir> -o <Catalyst_source_dir>
27
\end{minted}
Nicolas Vuaille's avatar
Nicolas Vuaille committed
28 29
Where \textless paraview\_sources\textgreater{} is the directory containing the ParaView sources and
\textless edition\_dir\textgreater{} is the directory containing the edition configuration files.
30
Note that more editions can be added with the -i \textless edition\_dir\textgreater \,and that these are processed in
31 32 33 34
the order they are given, first to last. For the minimal base edition included with ParaView, this
would be -i Editions/Base. The generated Catalyst source tree will be put in
\textless Catalyst\_source\_dir\textgreater. For configuring Catalyst from the desired build directory, do the
following:
35
\begin{minted}{python}
36
<Catalyst_source_dir>/cmake.sh <Catalyst_source_dir>
37
\end{minted}
38 39 40 41
The next step is to build Catalyst (e.g. using make on Linux systems).
\subsection{Creating a ParaView Catalyst Edition}
The main operations for creating an edition of Catalyst are:
\begin{enumerate}
Nicolas Vuaille's avatar
Nicolas Vuaille committed
42
\item Choose one or more editions to be the basis of the new one [optional]
43 44 45 46 47 48 49 50 51
\item Set CMake build parameters (e.g. static or shared library build).
\item Specify files from the ParaView source tree to be copied into the created Catalyst source tree.
\item Specify files from the edition to be copied into the Catalyst source tree.
\end{enumerate}

The information describing which files are in the generated Catalyst source tree is all stored in a
JSON file called manifest.json in the main directory of the edition. The user processes this
information with a Python script called catalyze.py that is located in the
\textless ParaView\_source\_dir\textgreater/Catalyst directory.
Nicolas Vuaille's avatar
Nicolas Vuaille committed
52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75

\subsection{Customizing editions}
Whereas creating a new edition from scratch may be quite complex, it is possible to extend
an existing edition and customize it. In order to properly separate the work, the customized part
of the edition has its own manifest.json file, that explicitly specifies which other editions are
required.

\begin{minted}{json}
{
  "edition": "Custom",
  "requires": [
    "Base",
    "Enable-Python"
  ]
}
\end{minted}
In this case, a "Base" edition and an "Enable-Python" edition must be provided when exporting
the Custom edition. The script will first process the "Base" configuration, then the "Enable-Python"
configuration and finally the "Custom" one. This allows an incremental workflow for creating editions.
Note that the \emph{-i} in command line options must be ordered, required editions appear first:
\begin{minted}{python}
python catalyze.py -r <paraview_sources> -i Path/to/Base -i Path/to/Enable-Python \
  -i Path/to/Custom -o <Catalyst_source_dir>
\end{minted}
76 77 78 79
\subsection{Setting CMake Build Parameters}
By default, Catalyst will be built with the default ParaView build parameters (e.g. build with
shared libraries) unless one of the Catalyst editions changes that in its manifest.json file. An
example of this is shown below:
80
\begin{minted}{json}
81
{
82 83 84 85 86 87 88 89 90 91
  "edition": "Custom",
  "cmake":{
    "cache":[
      {
        "name":"BUILD_SHARED_LIBS",
        "type":"BOOL",
        "value":"OFF"
      }
    ]
  }
92
}
93
\end{minted}
94
Here, ParaView's CMake option of building shared libraries will be set to OFF for this edition
95
named Custom. It should be
96 97 98 99 100 101 102 103 104
noted that users can still change the build configuration from these settings but it should be
done after Catalyst is configured with the cmake.sh script.
\subsection{Copying Files from the ParaView Source Tree into the Created Catalyst Source Tree}
By default, very little source code from the ParaView source tree will be copied to the generated
Catalyst source tree. Each edition will likely want to add in several source code files to the
Catalyst source tree. Most of these files will be filters but there may also be several helper
classes that are needed to be copied over as well. In the following JSON snippet we
demonstrate how to copy the vtkPVArrayCalculator class into the generated Catalyst source
tree.
105
\begin{minted}{json}
106
{
107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122
  "edition": "Custom",
  "modules":[
    {
      "name":"vtkPVVTKExtensionsDefault",
      "path":"ParaViewCore/VTKExtensions/Default"
      "include":[
        {
          "path":"vtkPVArrayCalculator.cxx"
        },
        {
          "path":"vtkPVArrayCalculator.h"
        }
      ],
      "cswrap":true
    }
  ]
123
}
124
\end{minted}
125 126
A description of the pertinent information follows:
\begin{itemize}
127
\item {\ttfamily "}name{\ttfamily "}:{\ttfamily "}vtkPVVTKExtensionsDefault{\ttfamily "} -- the name of the VTK or ParaView module.
128 129 130
In this case it is vtkPVVTKExtensionsDefault. The name of the module can be found
in the modules.cmake file in the corresponding directory. It is the first argument to the
vtk\_module() function.
131 132 133 134 135 136 137
\item {\ttfamily "}path{\ttfamily "}:{\ttfamily "}ParaViewCore/VTKExtensions/Default{\ttfamily "} --
the subdirectory location of the
module relative to the main source tree directory (e.g.
\textless ParaView\_source\_dir\textgreater/ParaViewCore/VTKExtensions/Default
in this case)
\item {\ttfamily "}path{\ttfamily "}:{\ttfamily "}vtkPVArrayCalculator.cxx{\ttfamily "} --
the name of the file to copy from the ParaView
138
source tree to the generated Catalyst source tree.
139
\item {\ttfamily "}cswrap{\ttfamily "}:true -- if the source code needs to be client-server wrapped such that it is
140 141 142
available through ParaView's server-manager. For filters that are used through
ParaView's Python interface or through a server-manager hard-coded \Cplusplus pipeline
this should be true. For helper classes this should be false.
143 144 145 146 147 148
\end{itemize}
The difficult part here is determining which files need to be included in Catalyst. In the example
above, the actual name of the ParaView proxy for the vtkPVArrayCalculator is Calculator. Thus,
to construct a ParaView client proxy for vtkPVArrayCalculator on the server, the user would
need to call Calculator() in the Python script. The best way to determine this connection
between the name of the ParaView proxy and the actual source code is in the XML files in the
149
ParaViewCore/ServerManager/SMApplication/Resources directory. In this case the proxy definition is in
150 151 152 153 154 155 156 157
the filters.xml file. The proxy label XML element will be converted into the Python constructor for
the proxy and the class name is stored in the proxy class XML element. The conversion of the
proxy label is done by removing spaces in the XML attribute. This is sufficient for many
situations but for some cases there will be additional classes needed to be included in order to
properly compile Catalyst. This can occur when the included source code derives from a class
not already included in Catalyst or uses helper classes not already included in Catalyst. For the
vtkPVArrayCalculator class we will also need to include the vtkArrayCalculator class that it
derives from.
158
\subsection{Copying Files From the Edition Into the Catalyst Source Tree}
159 160
Some of the files that need to be in the generated Catalyst source tree cannot be directly copied
over from the ParaView source tree. For example, CMakeLists.txt files need to be modified in
161 162
the Catalyst source tree when multiple editions need to be added into a specialized CMakeLists.txt
file in the same directory. This is done with the {\ttfamily "}replace{\ttfamily "} keyword. An example of this is shown
163 164 165
below for the vtkFiltersCore module. Here, the vtkArrayCalculator source code is added to the
Catalyst source tree and so the CMakeLists.txt file in that directory needs to be modified in
order to include that class to be added to the build.
166 167
\begin{minted}{json}
"modules":[
168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185
  {
    "name":"vtkFiltersCore",
    "path":"VTK/Filters/Core",
    "include":[
      {
        "path":"vtkArrayCalculator.cxx"
      },
      {
        "path":"vtkArrayCalculator.h"
      }
    ],
    "replace":[
      {
        "path":"VTK/Filters/Core/CMakeLists.txt"
      }
    ],
    "cswrap":true
  }
186
]
187
\end{minted}
188 189 190 191 192
In this case, the CMakeLists.txt file that needs to be copied to the Catalyst source tree exists in
the \textless edition\_dir\textgreater/VTK/Filters/Core directory, where edition\_dir is the location of this custom
edition of Catalyst. Since the Base edition already includes some files from this directory, we
want to make sure that the CMakeLists.txt file from this edition also includes those from the
Base edition. This CMakeLists.txt file is shown below:
193
\begin{minted}{cmake}
194
set(Module_SRCS
195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218
  vtkArrayCalculator.cxx
  vtkCellDataToPointData.cxx
  vtkContourFilter.cxx
  vtkContourGrid.cxx
  vtkContourHelper.cxx
  vtkCutter.cxx
  vtkExecutionTimer.cxx
  vtkFeatureEdges.cxx
  vtkGridSynchronizedTemplates3D.cxx
  vtkMarchingCubes.cxx
  vtkMarchingSquares.cxx
  vtkPointDataToCellData.cxx
  vtkPolyDataNormals.cxx
  vtkProbeFilter.cxx
  vtkQuadricClustering.cxx
  vtkRectilinearSynchronizedTemplates.cxx
  vtkSynchronizedTemplates2D.cxx
  vtkSynchronizedTemplates3D.cxx
  vtkSynchronizedTemplatesCutter3D.cxx
  vtkThreshold.cxx
  vtkAppendCompositeDataLeaves.cxx
  vtkAppendFilter.cxx
  vtkAppendPolyData.cxx
  vtkImageAppend.cxx
219 220
)
set_source_files_properties(
221 222
  vtkContourHelper
  WRAP_EXCLUDE
223 224
)
vtk_module_library(vtkFiltersCore ${Module_SRCS})
225
\end{minted}
226 227 228 229 230
Note that this CMakeLists.txt file does two things. Firstly it specifies which files to be compiled in
the source directory. Next, it specifies properties of the source files. In the above example,
vtkContourHelper is given a property specifying that it should not be wrapped. Another property
which is commonly set indicates that a class is an abstract class (i.e. it has pure virtual
functions). An example of how to do this is shown below.
231
\begin{minted}{cmake}
232
set_source_files_properties(
233 234 235
  vtkXMLPStructuredDataWriter
  vtkXMLStructuredDataWriter
  ABSTRACT
236
)
237
\end{minted}
Nicolas Vuaille's avatar
Nicolas Vuaille committed
238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261

\subsection{Exposing proxies}
To be available as a ParaView source, reader, filter or writer, a VTK class should be exposed through a Proxy.
This is done in XML files that described proxies and their properties, as explained in
\ref{subsection:cppPipeline}.
To extract new proxies from these decription files, they should be added under the
{\ttfamily "}proxies{\ttfamily "} keyword.
Below is an example for a filter and a source:
\begin{minted}{json}
"proxies":[
  {
    "path":"ParaViewCore/ServerManager/SMApplication/Resources/filters.xml",
    "proxies":[
      "Glyph"
    ]
  },
  {
    "path":"ParaViewCore/ServerManager/SMApplication/Resources/sources.xml",
    "proxies":[
      "ArrowSource"
    ]
  }
]
\end{minted}