High memory usage TUs
I've been seeing OOM situations arise more and more recently of late with VTK-m. This issue aims to document the high-usage memory TUs. Note that I do not have CUDA enabled, and memory usage tends to be higher with its toolchain.
Configuration: RelWithDebInfo, no CUDA, GCC 10.2.1. Running the build with systemd-run --user -p MemoryMax=$memory --setenv=CCACHE_DISABLE=1 --same-dir --pty ninja -j1 $target
TUs that fail with memory=4G
:
-
Accelerators/Vtkm/Filters/CMakeFiles/AcceleratorsVTKmFilters.dir/vtkmClip.cxx.o
-
Accelerators/Vtkm/Filters/CMakeFiles/AcceleratorsVTKmFilters.dir/vtkmClipWithField.cxx.o
-
Accelerators/Vtkm/Filters/CMakeFiles/AcceleratorsVTKmFilters.dir/vtkmGradient.cxx.o
Will update as things progress.
EDITED (Nov 23rd 2020)
Progress
-
[VTKm optimizations] in vtkm repo (!7390 (merged)) -
[vtkmClip optimizations] in vtk repo (!7397 (merged))
After the above tasks we were able to reduce the compilation memory usage in the following manner:
Version | Size sum of object files | Minimum memory to compile |
---|---|---|
Before | 129M | 5GiB |
1st MR | 74M | 4GiB |
2nd MR | 107MiB | 3GiB |
See: vtk-m#578 (closed)
Cc: @vbolea @robertmaynard