GitLab

Overview

Currently VTK-m has no easy way to efficiently integrate with VTK, allowing VTK-m worklets/filters to be called from inside a VTK pipeline.

The primary deficiency is that VTK-m has a worklet plus array handle view of execution and doesn't offer any way to integrate into VTK's pipeline and dataset model.

Therefore I propose the following solution.

• We add storage types to VTK-m that are memory layout compatible to VTK and offer helper methods that allow vtkm::cont::DataSet to use these specializations.

  This means that we would have the following:

  • vtkm::cont::VTKVolumeCellSet
  • vtkm::cont::VTKUnstructuredCellSet
  • vtkm::cont::VTKSingleTypeCellSet

  Each one of these new custom cellsets would be a derived version of vtkm::cont::CellSet that specifies the proper storage tags to vtkm::cont::CellSetExplicit. The end result will be VTK-m datasets that can re-use memory already allocated by VTK.

  Now the only issue with this approach is mapping vtkPoints into a vtkm::cont::DataSet + vtkm::cont::CoordinateSystem. I expect the easiest solution is that a field called "xyz" will be used to represent the VTK point coordinates while Jeremy,David, and Sujin determine a better solution.

• We add the concept of filters to VTK-m. In VTK filters are designed with a system where you pass a dataset in, and you receive a dataset out without any information on what has changed. For VTK-m I believe filters should report exactly what has changed.

  To do this I believe that filters should be designed around what kind of data they generate. This would result in the possible filter 'parent' types:

  • vtkm::filter::PointFieldFilter
  • vtkm::filter::CellFieldFilter
  • vtkm::filter::CellSetFilter
  • vtkm::filter::CoordinateFilter
  • vtkm::filter::DataSetFilter

  For example CellAverage is a filter that generates a cell based field, so it should inherit from CellFieldFilter as it needs both a DataSet as input and what CellSet to use for topology information. While something like ExternalFaces is a CellSetFilter as it generates an entire new CellSet from the input DataSet and CellSet.

By combing vtkm::cont::DataSet, derived vtkm::cont::CellSet with ability to pre-compile VTK-m filters, we have the ability to use VTK-m filters from within VTK. The requirement is that the VTK-m filter infrastructure has to cast the CellSet from a vtkm::cont::CellSet into the correct derived type. Than on the VTK set when wrap VTK-m filters we need to do a VTK to VTK-m data set conversion.

In the future we can add infrastructure to vtkm::filter for easier/automatic conversion of derived vtkm::cont::CellSets to concrete type. This would allow interop with VisIt or In-Situ codes without complication.

Technical Issues

1. How are you going to create a VTK compatible memory layout without linking to VTK?
   VTK-m will provide implementations of vtkm::cont::internal::Storage for the custom tags vtkCellArrayStorageTag and vtkFieldStorageTag. Each of these tags that will allocate memory exactly the same way VTK does and uses the vtkm Storage StealArray() functionality combined with vtkAbstractArray's SetVoidArray to transfer memory ownership.

2. How are you go to marshal VTK DataArrays into VTK-m?
   At first glance it seems fairly easy to access memory allocated by VTK inside VTK-m. All you need to do is get the raw underlying memory pointer (GetVoidPointer()) and pass that to vtkm::cont::ArrayHandle. The issues with this is that for VTK number of components and component type is run time information. So to solve this issue we need to expand all combinations when converting between VTK and VTK-m.
   For this to work properly it does mean that we need to create new type-list and storage-list tags for the DynamicArrayHandle that allows us to hold onto vtkDataArray's and convert to all supported number of components.
   Here is some example code to convert from vtkDataArray to a DynamicArrayHandle:

template<typename ValueType>
vtkm::cont::DynamicArrayHandle deduceNumberOfComponents(vtkDataArray* data)
{
  switch(data->GetNumberOfComponents())
  {
    case 1:
      vtkm::cont::ArrayHandle<ValueType> handle( vtkm::cont::vtk::make_vtkStorage(data, ValueType()) );
      return vtkm::cont::DynamicArrayHandle( handle );
      break;
    case 2:
      typedef vtkm::Vec<ValueType,2> VecType;
      vtkm::cont::ArrayHandle<VecType> handle( vtkm::cont::vtk::make_vtkStorage(data, VecType()) );
      return vtkm::cont::DynamicArrayHandle( handle );
      break;
    case 3:
      typedef vtkm::Vec<ValueType,3> VecType;
      vtkm::cont::ArrayHandle<VecType> handle( vtkm::cont::vtk::make_vtkStorage(data, VecType()) );
      return vtkm::cont::DynamicArrayHandle( handle );
      break;
    ...
  }
  return vtkm::cont::DynamicArrayHandle();
}
vtkm::cont::DynamicArrayHandle to_DynamicArrayHandle(vtkDataArray* data)
{
  vtkm::cont::DynamicArrayHandle result;
  switch(array->GetDataType())
  {
    vtkTemplateMacro( result = deduceNumberOfComponents(data) );
  }
  result->ResetTypeList( vtkm::cont::vtk::vtkComponentTypeList() );
  result->ResetStorageList( vtkm::cont::vtk::vtkStorageTypeList() );
  return result;
}
vtkDataArray* foo = input->GetPointData()->GetArray("temp");
vtkm::cont::DynamicArrayHandle converted = to_DynamicArrayHandle( foo );

3. How are you going to marshal VTK DataSet's into VTK-m?
   Once we have the ability to convert data-arrays between VTK and VTK-m, the rest is fairly easy. The logic to convert Unstructured VTK data to VTK-m is even easier as the vtkCellArray types are known at compile time, and the Points array is easy ( types are float / double, number of components are always 3).

Example CellFieldFilter

class CellFieldFilter
{
public:
  virtual vtkm::cont::Field Execute(vtkm::cont::DataSet &input,
                                    vtkm::Id cellSetId,
                                    const std::string &inFieldName);
}

class CellAverage : public vtkm::filter::CellFieldFilter
{
public:
  vtkm::cont::Field Execute(vtkm::cont::DataSet &input,
                            vtkm::Id cellSetId,
                            const std::string &inFieldName)
  {
  ...
  return vtkm::cont::Field();
  }

};

Note: Edited based on the discussion with @kmorel about how to implement VTK to DynamicArrayHandle.