BuildSection.tex 12 KB
 Andrew Bauer committed Dec 04, 2014 1 2 3 4 5 6 7 8 9 10 11 12 13 14 \section{Building ParaView Catalyst} This section is targeted towards those users or developers responsible for building ParaView Catalyst. As far as installation is concerned, Catalyst is a subset of the ParaView code base. Thus, all of the functionality available in Catalyst is also available in ParaView. The difference is that ParaView will by default have many more dependencies and thus will have a larger executable size. Catalyst is the flexible and specialized configuration of ParaView that is used to reduce the executable size by reducing dependencies. For example, if no output images are to be produced from a Catalyst-instrumented simulation run then all of the ParaView and VTK code related to rendering and the OpenGL libraries need not be linked in. This can result in significant memory savings, especially when considering the number of processes utilized when running a simulation at scale. In one simple example, the executable size was reduced from 75 MB when linking with ParaView to less than 20 MB when linking with Catalyst. The main steps for configuring Catalyst are: \begin{enumerate}  Andrew Bauer committed Sep 30, 2016 15 \item Set up an “edition”  Andrew Bauer committed Dec 04, 2014 16 17 18 19 20 21 22 23 24 \item Extract the desired code from ParaView source tree into a separate Catalyst source tree \item Build Catalyst \end{enumerate} Most of the work is in the first step which is described below. A Catalyst edition is a customization of ParaView to support a desired subset of functionality from ParaView and VTK. There can be many editions of Catalyst and these editions can be combined to create several customized Catalyst builds. Assuming that the desired editions have already been created, the second step is automated and is done by invoking the following command from the \textless ParaView\_source\_dir\textgreater /Catalyst directory:  Andrew Bauer committed Jan 26, 2015 25 \begin{minted}{python}  Nicolas Vuaille committed Aug 03, 2018 26 python catalyze.py -r -i -o  Andrew Bauer committed Jan 26, 2015 27 \end{minted}  Nicolas Vuaille committed Aug 03, 2018 28 29 Where \textless paraview\_sources\textgreater{} is the directory containing the ParaView sources and \textless edition\_dir\textgreater{} is the directory containing the edition configuration files.  Andrew Bauer committed Feb 19, 2015 30 Note that more editions can be added with the -i \textless edition\_dir\textgreater \,and that these are processed in  Andrew Bauer committed Dec 04, 2014 31 32 33 34 the order they are given, first to last. For the minimal base edition included with ParaView, this would be -i Editions/Base. The generated Catalyst source tree will be put in \textless Catalyst\_source\_dir\textgreater. For configuring Catalyst from the desired build directory, do the following:  Andrew Bauer committed Jan 26, 2015 35 \begin{minted}{python}  Andrew Bauer committed Dec 04, 2014 36 /cmake.sh  Andrew Bauer committed Jan 26, 2015 37 \end{minted}  Andrew Bauer committed Dec 04, 2014 38 39 40 41 The next step is to build Catalyst (e.g. using make on Linux systems). \subsection{Creating a ParaView Catalyst Edition} The main operations for creating an edition of Catalyst are: \begin{enumerate}  Nicolas Vuaille committed Aug 03, 2018 42 \item Choose one or more editions to be the basis of the new one [optional]  Andrew Bauer committed Dec 04, 2014 43 44 45 46 47 48 49 50 51 \item Set CMake build parameters (e.g. static or shared library build). \item Specify files from the ParaView source tree to be copied into the created Catalyst source tree. \item Specify files from the edition to be copied into the Catalyst source tree. \end{enumerate} The information describing which files are in the generated Catalyst source tree is all stored in a JSON file called manifest.json in the main directory of the edition. The user processes this information with a Python script called catalyze.py that is located in the \textless ParaView\_source\_dir\textgreater/Catalyst directory.  Nicolas Vuaille committed Aug 03, 2018 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75  \subsection{Customizing editions} Whereas creating a new edition from scratch may be quite complex, it is possible to extend an existing edition and customize it. In order to properly separate the work, the customized part of the edition has its own manifest.json file, that explicitly specifies which other editions are required. \begin{minted}{json} { "edition": "Custom", "requires": [ "Base", "Enable-Python" ] } \end{minted} In this case, a "Base" edition and an "Enable-Python" edition must be provided when exporting the Custom edition. The script will first process the "Base" configuration, then the "Enable-Python" configuration and finally the "Custom" one. This allows an incremental workflow for creating editions. Note that the \emph{-i} in command line options must be ordered, required editions appear first: \begin{minted}{python} python catalyze.py -r -i Path/to/Base -i Path/to/Enable-Python \ -i Path/to/Custom -o \end{minted}  Andrew Bauer committed Dec 04, 2014 76 77 78 79 \subsection{Setting CMake Build Parameters} By default, Catalyst will be built with the default ParaView build parameters (e.g. build with shared libraries) unless one of the Catalyst editions changes that in its manifest.json file. An example of this is shown below:  Andrew Bauer committed Jan 26, 2015 80 \begin{minted}{json}  Andrew Bauer committed Dec 04, 2014 81 {  Andrew Bauer committed Jan 26, 2015 82 83 84 85 86 87 88 89 90 91  "edition": "Custom", "cmake":{ "cache":[ { "name":"BUILD_SHARED_LIBS", "type":"BOOL", "value":"OFF" } ] }  Andrew Bauer committed Dec 04, 2014 92 }  Andrew Bauer committed Jan 26, 2015 93 \end{minted}  Andrew Bauer committed Feb 19, 2015 94 Here, ParaView's CMake option of building shared libraries will be set to OFF for this edition  Andrew Bauer committed Jan 26, 2015 95 named Custom. It should be  Andrew Bauer committed Dec 04, 2014 96 97 98 99 100 101 102 103 104 noted that users can still change the build configuration from these settings but it should be done after Catalyst is configured with the cmake.sh script. \subsection{Copying Files from the ParaView Source Tree into the Created Catalyst Source Tree} By default, very little source code from the ParaView source tree will be copied to the generated Catalyst source tree. Each edition will likely want to add in several source code files to the Catalyst source tree. Most of these files will be filters but there may also be several helper classes that are needed to be copied over as well. In the following JSON snippet we demonstrate how to copy the vtkPVArrayCalculator class into the generated Catalyst source tree.  Andrew Bauer committed Jan 26, 2015 105 \begin{minted}{json}  Andrew Bauer committed Dec 04, 2014 106 {  Andrew Bauer committed Jan 26, 2015 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122  "edition": "Custom", "modules":[ { "name":"vtkPVVTKExtensionsDefault", "path":"ParaViewCore/VTKExtensions/Default" "include":[ { "path":"vtkPVArrayCalculator.cxx" }, { "path":"vtkPVArrayCalculator.h" } ], "cswrap":true } ]  Andrew Bauer committed Dec 04, 2014 123 }  Andrew Bauer committed Jan 26, 2015 124 \end{minted}  Andrew Bauer committed Dec 04, 2014 125 126 A description of the pertinent information follows: \begin{itemize}  Andrew Bauer committed Jan 26, 2015 127 \item {\ttfamily "}name{\ttfamily "}:{\ttfamily "}vtkPVVTKExtensionsDefault{\ttfamily "} -- the name of the VTK or ParaView module.  Andrew Bauer committed Dec 04, 2014 128 129 130 In this case it is vtkPVVTKExtensionsDefault. The name of the module can be found in the modules.cmake file in the corresponding directory. It is the first argument to the vtk\_module() function.  Andrew Bauer committed Jan 26, 2015 131 132 133 134 135 136 137 \item {\ttfamily "}path{\ttfamily "}:{\ttfamily "}ParaViewCore/VTKExtensions/Default{\ttfamily "} -- the subdirectory location of the module relative to the main source tree directory (e.g. \textless ParaView\_source\_dir\textgreater/ParaViewCore/VTKExtensions/Default in this case) \item {\ttfamily "}path{\ttfamily "}:{\ttfamily "}vtkPVArrayCalculator.cxx{\ttfamily "} -- the name of the file to copy from the ParaView  Andrew Bauer committed Dec 04, 2014 138 source tree to the generated Catalyst source tree.  Andrew Bauer committed Jan 26, 2015 139 \item {\ttfamily "}cswrap{\ttfamily "}:true -- if the source code needs to be client-server wrapped such that it is  Andrew Bauer committed Dec 08, 2014 140 141 142 available through ParaView's server-manager. For filters that are used through ParaView's Python interface or through a server-manager hard-coded \Cplusplus pipeline this should be true. For helper classes this should be false.  Andrew Bauer committed Dec 04, 2014 143 144 145 146 147 148 \end{itemize} The difficult part here is determining which files need to be included in Catalyst. In the example above, the actual name of the ParaView proxy for the vtkPVArrayCalculator is Calculator. Thus, to construct a ParaView client proxy for vtkPVArrayCalculator on the server, the user would need to call Calculator() in the Python script. The best way to determine this connection between the name of the ParaView proxy and the actual source code is in the XML files in the  Andrew Bauer committed Dec 08, 2014 149 ParaViewCore/ServerManager/SMApplication/Resources directory. In this case the proxy definition is in  Andrew Bauer committed Dec 04, 2014 150 151 152 153 154 155 156 157 the filters.xml file. The proxy label XML element will be converted into the Python constructor for the proxy and the class name is stored in the proxy class XML element. The conversion of the proxy label is done by removing spaces in the XML attribute. This is sufficient for many situations but for some cases there will be additional classes needed to be included in order to properly compile Catalyst. This can occur when the included source code derives from a class not already included in Catalyst or uses helper classes not already included in Catalyst. For the vtkPVArrayCalculator class we will also need to include the vtkArrayCalculator class that it derives from.  Andrew Bauer committed Feb 19, 2015 158 \subsection{Copying Files From the Edition Into the Catalyst Source Tree}  Andrew Bauer committed Dec 04, 2014 159 160 Some of the files that need to be in the generated Catalyst source tree cannot be directly copied over from the ParaView source tree. For example, CMakeLists.txt files need to be modified in  Andrew Bauer committed Feb 19, 2015 161 162 the Catalyst source tree when multiple editions need to be added into a specialized CMakeLists.txt file in the same directory. This is done with the {\ttfamily "}replace{\ttfamily "} keyword. An example of this is shown  Andrew Bauer committed Dec 04, 2014 163 164 165 below for the vtkFiltersCore module. Here, the vtkArrayCalculator source code is added to the Catalyst source tree and so the CMakeLists.txt file in that directory needs to be modified in order to include that class to be added to the build.  Andrew Bauer committed Jan 26, 2015 166 167 \begin{minted}{json} "modules":[  Andrew Bauer committed Sep 30, 2016 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185  { "name":"vtkFiltersCore", "path":"VTK/Filters/Core", "include":[ { "path":"vtkArrayCalculator.cxx" }, { "path":"vtkArrayCalculator.h" } ], "replace":[ { "path":"VTK/Filters/Core/CMakeLists.txt" } ], "cswrap":true }  Andrew Bauer committed Dec 04, 2014 186 ]  Andrew Bauer committed Jan 26, 2015 187 \end{minted}  Andrew Bauer committed Dec 04, 2014 188 189 190 191 192 In this case, the CMakeLists.txt file that needs to be copied to the Catalyst source tree exists in the \textless edition\_dir\textgreater/VTK/Filters/Core directory, where edition\_dir is the location of this custom edition of Catalyst. Since the Base edition already includes some files from this directory, we want to make sure that the CMakeLists.txt file from this edition also includes those from the Base edition. This CMakeLists.txt file is shown below:  Andrew Bauer committed Jan 26, 2015 193 \begin{minted}{cmake}  Andrew Bauer committed Dec 04, 2014 194 set(Module_SRCS  Andrew Bauer committed Jan 26, 2015 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218  vtkArrayCalculator.cxx vtkCellDataToPointData.cxx vtkContourFilter.cxx vtkContourGrid.cxx vtkContourHelper.cxx vtkCutter.cxx vtkExecutionTimer.cxx vtkFeatureEdges.cxx vtkGridSynchronizedTemplates3D.cxx vtkMarchingCubes.cxx vtkMarchingSquares.cxx vtkPointDataToCellData.cxx vtkPolyDataNormals.cxx vtkProbeFilter.cxx vtkQuadricClustering.cxx vtkRectilinearSynchronizedTemplates.cxx vtkSynchronizedTemplates2D.cxx vtkSynchronizedTemplates3D.cxx vtkSynchronizedTemplatesCutter3D.cxx vtkThreshold.cxx vtkAppendCompositeDataLeaves.cxx vtkAppendFilter.cxx vtkAppendPolyData.cxx vtkImageAppend.cxx  Andrew Bauer committed Dec 04, 2014 219 220 ) set_source_files_properties(  Andrew Bauer committed Jan 26, 2015 221 222  vtkContourHelper WRAP_EXCLUDE  Andrew Bauer committed Dec 04, 2014 223 224 ) vtk_module_library(vtkFiltersCore \${Module_SRCS})  Andrew Bauer committed Jan 26, 2015 225 \end{minted}  Andrew Bauer committed Dec 04, 2014 226 227 228 229 230 Note that this CMakeLists.txt file does two things. Firstly it specifies which files to be compiled in the source directory. Next, it specifies properties of the source files. In the above example, vtkContourHelper is given a property specifying that it should not be wrapped. Another property which is commonly set indicates that a class is an abstract class (i.e. it has pure virtual functions). An example of how to do this is shown below.  Andrew Bauer committed Jan 26, 2015 231 \begin{minted}{cmake}  Andrew Bauer committed Dec 04, 2014 232 set_source_files_properties(  Andrew Bauer committed Jan 26, 2015 233 234 235  vtkXMLPStructuredDataWriter vtkXMLStructuredDataWriter ABSTRACT  Andrew Bauer committed Dec 04, 2014 236 )  Andrew Bauer committed Jan 26, 2015 237 \end{minted}  Nicolas Vuaille committed Aug 03, 2018 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261  \subsection{Exposing proxies} To be available as a ParaView source, reader, filter or writer, a VTK class should be exposed through a Proxy. This is done in XML files that described proxies and their properties, as explained in \ref{subsection:cppPipeline}. To extract new proxies from these decription files, they should be added under the {\ttfamily "}proxies{\ttfamily "} keyword. Below is an example for a filter and a source: \begin{minted}{json} "proxies":[ { "path":"ParaViewCore/ServerManager/SMApplication/Resources/filters.xml", "proxies":[ "Glyph" ] }, { "path":"ParaViewCore/ServerManager/SMApplication/Resources/sources.xml", "proxies":[ "ArrowSource" ] } ] \end{minted}