ADIOS2: Add SZ support
I would like to reduce our data footprint for transient CFD fielddata and therefore intend to write data lossily compressed data.
Since HDF5 does not support the lossy compression algorithms that are actually interesting I found the Adios2 BP4 data files promising. They support both ZFP and SZ compressions, among many others.
Now, the ADIOS2CoreImageReader
(and hopefully also the ADIOS2VTXReader
) actually supports ZFP-compressed data out of the box, which is great, but it fails for SZ-compressed data, stating that Adios2 was compiled without SZ.
Looking at projects/adios2.cmake
in paraview-superbuild I conclude that SZ-support is not yet implemented:
-DADIOS2_USE_SZ:STRING=OFF
-DADIOS2_USE_ZFP:STRING=${zfp_enabled}
Is it feasable to add support for SZ? Are there known caveats that have prevented you from doing it?
I guess one would have to:
- add
projects/sz.cmake
andprojects/sz.system.cmake
- modify
projects/adios2.cmake
making use of${sz_enabled}
- add
sz
toCMakeLists.txt
- extend
versions.cmake
maybe also:
- add
set(ENABLE_sz ON CACHE BOOL "")
to.gitlab/ci/configure_common.cmake
- maipulate
.gitlab/gitlab-ci-olcf.yml
to include-DENABLE_sz:BOOL=ON
Based on how I currently compile SZ, zlib must also be enabled in order to enable SZ.
For usage in our CFD solver, I currently compile SZ like so:
cmake -DBUILD_FORTRAN=ON \
-DCMAKE_C_COMPILER=$CC \
-DCMAKE_CXX_COMPILER=$CXX \
-DCMAKE_Fortran_COMPILER=$FC \
-DZLIB_ROOT="${ZLIB_ROOT}" \
-DCMAKE_INSTALL_PREFIX:PATH=${SZ_ROOT} ${dir_src}
make
make install
and then Adios2
cmake -DCMAKE_C_COMPILER=$CC \
-DCMAKE_CXX_COMPILER=$CXX \
-DCMAKE_Fortran_COMPILER=$FC \
-DSZ_ROOT=$SZ_ROOT \
-DZFP_ROOT=$ZFP_ROOT \
-DHDF5_ROOT=$HDF5_ROOT \
-DCMAKE_INSTALL_PREFIX:PATH=${ADIOS2_ROOT} \
$dir_src
make -j 16
ctest
make install