VTK-m merge requestshttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests2023-04-03T15:30:41-04:00https://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/3004Simplify serialization of DataSet objects2023-04-03T15:30:41-04:00Kenneth MorelandSimplify serialization of DataSet objects`vtkm::cont::DataSet` is a dynamic object that can hold cell sets and
fields of many different types, none of which are known until runtime. This
causes a problem with serialization, which has to know what type to compile
the serializati...`vtkm::cont::DataSet` is a dynamic object that can hold cell sets and
fields of many different types, none of which are known until runtime. This
causes a problem with serialization, which has to know what type to compile
the serialization for, particularly when unserializing the type at the
receiving end. The original implementation "solved" the problem by creating
a secondary wrapper object that was templated on types of field arrays and
cell sets that might be serialized. This is not a great solution as it
punts the problem to algorithm developers.
This problem has been completely solved for fields, as it is possible to
serialize most types of arrays without knowing their type now. You still
need to iterate over every possible `CellSet` type, but there are not that
many `CellSet`s that are practically encountered. Thus, there is now a
direct implementation of `Serialization` for `DataSet` that covers all the
data types you are likely to encounter.
The old `SerializableDataSet` has been deprecated. In the unlikely event an
algorithm needs to transfer a non-standard type of `CellSet` (such as a
permuted cell set), it can use the replacement `DataSetWithCellSetTypes`,
which just specifies the cell set types.
Fixes #725https://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/3003ascent,ci,olcf: split build and test stage2023-04-03T15:30:45-04:00Vicente Boleavicente.bolea@kitware.comascent,ci,olcf: split build and test stage- It appears that the recurrent error in the Ascent build about missing files is created by a race condition of multiple builds introduced in a recent MR which attempted to optimize the build.
- That change consisted in separating the c...- It appears that the recurrent error in the Ascent build about missing files is created by a race condition of multiple builds introduced in a recent MR which attempted to optimize the build.
- That change consisted in separating the configure phase in a single job and build/test in another. This caused the race condition since the configure phase sets its source_path which is accessed during the build phase, however, when this happens that path might have been reused by another concurrent machine, erroring in unexpected ways.
- I reverted to the previous way of keeping configure/build in a single stage and test in another one. I also added some desirable changes to to this MR to further tune the build.Vicente Boleavicente.bolea@kitware.comVicente Boleavicente.bolea@kitware.comhttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/3002Spock build kokkos2023-04-03T15:30:45-04:00Vicente Boleavicente.bolea@kitware.comSpock build kokkosVicente Boleavicente.bolea@kitware.comVicente Boleavicente.bolea@kitware.comhttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/3000readme:add spack and gitlab ver badge2023-04-03T15:30:44-04:00Vicente Boleavicente.bolea@kitware.comreadme:add spack and gitlab ver badge@kmorel adding version badges to the readme file, the spack one perhaps is the most useful.@kmorel adding version badges to the readme file, the spack one perhaps is the most useful.Vicente Boleavicente.bolea@kitware.comVicente Boleavicente.bolea@kitware.comhttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2999cxx,c++: Remove c++14 references2023-04-03T15:30:44-04:00Vicente Boleavicente.bolea@kitware.comcxx,c++: Remove c++14 references- MSVC C++14 support obtained from: https://learn.microsoft.com/en-us/cpp/overview/visual-cpp-language-conformance?view=msvc-170
- _MSVC_VER taken from https://stackoverflow.com/a/70630/2420872
fixes: #754- MSVC C++14 support obtained from: https://learn.microsoft.com/en-us/cpp/overview/visual-cpp-language-conformance?view=msvc-170
- _MSVC_VER taken from https://stackoverflow.com/a/70630/2420872
fixes: #754Vicente Boleavicente.bolea@kitware.comVicente Boleavicente.bolea@kitware.comhttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2998release,docs: add freeze release branch step2023-04-03T15:30:45-04:00Vicente Boleavicente.bolea@kitware.comrelease,docs: add freeze release branch stepfixes: #760fixes: #760Vicente Boleavicente.bolea@kitware.comVicente Boleavicente.bolea@kitware.comhttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2997ascent,ci: reduce build concurrency level2023-04-03T15:30:45-04:00Vicente Boleavicente.bolea@kitware.comascent,ci: reduce build concurrency levelWe are observing many build failures n the ascent build. It appears to be some sort of resource exhaustion in the form of memory used, disk usage, number of inodes...
Most possibly can be something about the memory usage, we can defini...We are observing many build failures n the ascent build. It appears to be some sort of resource exhaustion in the form of memory used, disk usage, number of inodes...
Most possibly can be something about the memory usage, we can definitely reduce this by reducing the concurrency level of the build.
fixes: #757Vicente Boleavicente.bolea@kitware.comVicente Boleavicente.bolea@kitware.comhttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2996Update roadmap2023-04-03T15:30:44-04:00Vicente Boleavicente.bolea@kitware.comUpdate roadmapWhat do you think about these tentative release plan?What do you think about these tentative release plan?Vicente Boleavicente.bolea@kitware.comVicente Boleavicente.bolea@kitware.comhttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2995Update tutorial CMakeLists to reflect VTK-m target namespace changes2023-02-24T08:50:46-05:00Louis GombertUpdate tutorial CMakeLists to reflect VTK-m target namespace changesFixes !2939, the target namespaces were not updatedFixes !2939, the target namespaces were not updatedhttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2994Automatically convert between ArrayHandleBasic and ArrayHandleRuntimeVec2023-04-03T15:30:41-04:00Kenneth MorelandAutomatically convert between ArrayHandleBasic and ArrayHandleRuntimeVecThe `UnknownArrayHandle` has been updated to allow
`ArrayHandleRuntimeVec` to work interchangeably with basic
`ArrayHandle`. If an `ArrayHandleRuntimeVec` is put into an
`UnknownArrayHandle`, it can be later retrieved as an `ArrayHandleB...The `UnknownArrayHandle` has been updated to allow
`ArrayHandleRuntimeVec` to work interchangeably with basic
`ArrayHandle`. If an `ArrayHandleRuntimeVec` is put into an
`UnknownArrayHandle`, it can be later retrieved as an `ArrayHandleBasic`
as long as the base component type matches and it has the correct amount
of components. This means that an array can be created as an
`ArrayHandleRuntimeVec` and be used with any filters or most other
features designed to operate on basic `ArrayHandle`s. Likewise, an array
added as a basic `ArrayHandle` can be retrieved in an
`ArrayHandleRuntimeVec`. This makes it easier to pull arrays from VTK-m
and place them in external structures (such as `vtkDataArray`).
Fixes #752https://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2991kokkos: disable volatile when kokkos >= 3.72023-04-03T15:30:44-04:00Vicente Boleavicente.bolea@kitware.comkokkos: disable volatile when kokkos >= 3.7Followup: !2986Followup: !2986Vicente Boleavicente.bolea@kitware.comVicente Boleavicente.bolea@kitware.comhttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2990Compile UnitTestAbort.cxx with a normal C++ compiler2023-04-03T15:30:45-04:00Kenneth MorelandCompile UnitTestAbort.cxx with a normal C++ compiler`UnitTestAbort.cxx` does not touch any code that needs to be compiled on
the device, so you do not need a device compiler. Use the standard C++
compiler instead.`UnitTestAbort.cxx` does not touch any code that needs to be compiled on
the device, so you do not need a device compiler. Use the standard C++
compiler instead.https://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2988Branch for the vupdate-to-2.0.0 release2023-02-20T13:27:33-05:00VTK-m Release ProcessBranch for the vupdate-to-2.0.0 release
---
Cc: @ben.boeckel @kmorel @vbolea
Fast-forward: true
Backport: master:HEAD~1
---
Cc: @ben.boeckel @kmorel @vbolea
Fast-forward: true
Backport: master:HEAD~1Vicente Boleavicente.bolea@kitware.comVicente Boleavicente.bolea@kitware.comhttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2987Enable GPU-to-GPU communication2023-07-07T13:27:53-04:00Vicente Boleavicente.bolea@kitware.comEnable GPU-to-GPU communicationDepends: !3024
This MR adds the flag `VTKm_ENABLE_GPU_MPI` which when enabled, it will use GPU aware MPI.
- This will only work with GPUs and MPI implementation that supports GPU aware MPI calls.
- Enabling `VTKm_ENABLE_GPU_MP` without...Depends: !3024
This MR adds the flag `VTKm_ENABLE_GPU_MPI` which when enabled, it will use GPU aware MPI.
- This will only work with GPUs and MPI implementation that supports GPU aware MPI calls.
- Enabling `VTKm_ENABLE_GPU_MP` without MPI/GPU support might results in errors when running VTK-m with DIY/MPI.
- Only the following tests can run with this feature if enabled:
- UnitTestSerializationDataSet
- UnitTestSerializationArrayHandle
Options used in crusher:
Followed OLCF docs about GPU aware MPI: https://docs.olcf.ornl.gov/systems/crusher_quick_start_guide.html#gpu-aware-mpi
CMake options:
```lang=cmake
set(CMAKE_BUILD_TYPE "debug" CACHE STRING "")
set(CMAKE_PREFIX_PATH /gpfs/alpine/csc331/scratch/vbolea/gpu-to-gpu/kokkos_install CACHE FILEPATH "")
#set(CMAKE_C_COMPILER /opt/rocm-4.5.0/llvm/bin/clang CACHE FILEPATH "")
#set(CMAKE_CXX_COMPILER /opt/rocm-4.5.0/llvm/bin/clang++ CACHE FILEPATH "")
set(CMAKE_HIP_FLAGS "-O0 -I/opt/cray/pe/mpich/8.1.14/ofi/crayclang/10.0/include" CACHE STRING "")
set(CMAKE_CXX_FLAGS "-O0 -I/opt/cray/pe/mpich/8.1.14/ofi/crayclang/10.0/include" CACHE STRING "")
set(VTKm_ENABLE_KOKKOS ON CACHE STRING "")
set(VTKm_ENABLE_MPI ON CACHE STRING "")
set(VTKm_ENABLE_GPU_MPI ON CACHE STRING "")
set(CMAKE_HIP_FLAGS "-O0" CACHE STRING "")
set(Kokkos_CXX_COMPILER /opt/rocm-4.5.0/llvm/bin/clang++ CACHE FILEPATH "")
set(Kokkos_COMPILE_LAUNCHER /opt/rocm-4.5.0/bin/amdclang++ CACHE FILEPATH "")
set(CMAKE_POSITION_INDEPENDENT_CODE ON CACHE BOOL "")
set(CMAKE_HIP_ARCHITECTURES "gfx90a" CACHE STRING "")
set(CMAKE_EXE_LINKER_FLAGS "-L/opt/cray/pe/mpich/8.1.14/ofi/crayclang/10.0/lib -lmpi -L/opt/cray/pe/mpich/8.1.14/gtl/lib -lmpi_gtl_hsa" CACHE STRING "")
set(MPIEXEC_EXECUTABLE /usr/bin/srun CACHE FILEPATH "")
set(MPIEXEC_PREFLAGS "-ACSC331;-pbatch;-t110;--nice=0;-N;1;--gpus-per-node;1;-c;1" CACHE STRING "")
```Vicente Boleavicente.bolea@kitware.comVicente Boleavicente.bolea@kitware.comhttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2986Changes for compiling with SYCL2023-04-03T15:30:43-04:00Mark BolstadChanges for compiling with SYCL- Remove volatile keyword from `join'
- Remove various `printf' in device code
Backport: release- Remove volatile keyword from `join'
- Remove various `printf' in device code
Backport: releasehttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2985Fix the passing of fields in filters2023-04-03T15:30:45-04:00Kenneth MorelandFix the passing of fields in filtersWhile updating the user's guide, I noticed a couple of minor problems
with how filters map fields. First, if a filter was using
`CreateResultCoordinateSystem`, it did not respect the
`PassCoordinateSystems` flag. Second, if both an `init...While updating the user's guide, I noticed a couple of minor problems
with how filters map fields. First, if a filter was using
`CreateResultCoordinateSystem`, it did not respect the
`PassCoordinateSystems` flag. Second, if both an `initializer_list` and
a mode was given to `SetFieldsToPass`, the mode was captured
incorrectly. Both problems are corrected.
Backport: releasehttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2984Fix unreferenced local variable warning2023-04-03T15:30:45-04:00Sujin PhilipFix unreferenced local variable warningbackport: releasebackport: releaseSujin PhilipSujin Philiphttps://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2983Add support for getting vec sizes of unknown arrays when runtime selected2023-04-03T15:30:41-04:00Kenneth MorelandAdd support for getting vec sizes of unknown arrays when runtime selectedThe `GetNumberOfComponents` and `GetNumberOfComponentsFlat` methods in
`UnknownArrayHandle` have been updated to correctly report the number of
components in special `ArrayHandle`s where the `Vec` sizes of the values
are not selected unt...The `GetNumberOfComponents` and `GetNumberOfComponentsFlat` methods in
`UnknownArrayHandle` have been updated to correctly report the number of
components in special `ArrayHandle`s where the `Vec` sizes of the values
are not selected until runtime.
Previously, these methods always reported 0 because the value type could
not report the size of the `Vec`. The lookup has been modified to query the
`ArrayHandle`'s `Storage` for the number of components where supported.
Note that this only works on `Storage` that provides a method to get the
runtime `Vec` size. If that is not provided, as will be the case if the
number of components can vary from one value to the next, it will still
report 0.
This feature is implemented by looking for a method named
`GetNumberOfComponents` is the `Storage` class for the `ArrayHandle`. If
this method is found, it is used to query the size at runtime.https://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2982Add ArrayHandleRuntimeVec2023-04-03T15:30:41-04:00Kenneth MorelandAdd ArrayHandleRuntimeVecThe new `ArrayHandleRuntimeVec` is a fancy `ArrayHandle` allows you to
specify a basic array of `Vec`s where the number of components of the `Vec`
are not known until runtime. (It can also optionally specify scalars.) The
behavior is muc...The new `ArrayHandleRuntimeVec` is a fancy `ArrayHandle` allows you to
specify a basic array of `Vec`s where the number of components of the `Vec`
are not known until runtime. (It can also optionally specify scalars.) The
behavior is much like that of `ArrayHandleGroupVecVariable` except that its
representation is much more constrained. This constrained representation
allows it to be automatically converted to an `ArrayHandleBasic` with the
proper `Vec` value type. This allows one part of code (such as a file
reader) to create an array with any `Vec` size, and then that array can be
fed to an algorithm that expects an `ArrayHandleBasic` of a certain value
type.https://gitlab.kitware.com/vtk/vtk-m/-/merge_requests/2981Change VectorMagnitude to work with any type of field2023-04-03T15:30:41-04:00Kenneth MorelandChange VectorMagnitude to work with any type of fieldPreviously, `VectorMagnitude` only worked with `Vec`s of size 2, 3, or
4. It now works with `Vec`s of any size. It also avoids a memory copy of
non-float types (although it does add a little arithmetic in the
computation).Previously, `VectorMagnitude` only worked with `Vec`s of size 2, 3, or
4. It now works with `Vec`s of any size. It also avoids a memory copy of
non-float types (although it does add a little arithmetic in the
computation).