Skip to content
Snippets Groups Projects

vtkHDFReader: add composite dataset support

Merged Lucas Givord requested to merge lucas.givord/vtk:vtkhdf/add_composite_support into master

Context

vtkHDF file format doesn't support for now composite dataset such as partitioned dataset collection, this MR intends to support it.

Solutions

To support it, we proposed a new spec for composite dataset, initially done in discourse, the final version is available here.

unfold this section to see the global schema for the composite dataset file format

image

Technical part

  • The documentation has been updated to fit the new file format for composite dataset.

  • The most important thing in this design is the Assembly group used to retrieve the hierarchy. Then we can, based on the hierarchy, append each already supported data type by calling the correct Read method.

  • vtkTestUtilities has been improved to be able to check each block inside a pdc

  • h5py scripts used to generate vtkHDF file in the context of this work is available in this repository : https://gitlab.kitware.com/keu-public/vtkhdf-scripts

Edited by Lucas Givord

Merge request reports

Merge request pipeline #379197 passed

Merge request pipeline passed for 391d56b8

Merged by Kitware RobotKitware Robot 1 year ago (Dec 15, 2023 7:52am UTC)

Loading

Pipeline #379281 passed

Pipeline passed for c88204a6 on master

7 environments impacted.

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Errors:

    • commit 165784ea is not allowed because the following files are not formatted according to the 'clang-format' check: Testing/Core/Testing/Cxx/TestDataObjectCompare.cxx, Testing/Core/vtkTestUtilities.cxx, Testing/Core/vtkTestUtilities.h. Post a comment ending in the line Do: reformat to rewrite the MR source branch automatically.
    • commit 2b900b4c is not allowed because the following files are not formatted according to the 'clang-format' check: IO/HDF/vtkHDFReader.cxx, IO/HDF/vtkHDFReader.h. Post a comment ending in the line Do: reformat to rewrite the MR source branch automatically.

    Warnings:

    • please consider adding a changelog entry in a file ending with .md in Documentation/release/dev.
    • the merge request is marked as a work-in-progress.

    The warnings do not need to be fixed, but it is recommended to do so.

    Please rewrite commits to fix the errors listed above (adding fixup commits will not resolve the errors) and force-push the branch again to update the merge request.

  • Lucas Givord added 2 commits

    added 2 commits

    • 76adb175 - vtkHDFReader: add support for composite dataset
    • 68213b6d - vtkHDFReader: add release note about composite support

    Compare with previous version

  • Errors:

    • commit 165784ea is not allowed because the following files are not formatted according to the 'clang-format' check: Testing/Core/Testing/Cxx/TestDataObjectCompare.cxx, Testing/Core/vtkTestUtilities.cxx, Testing/Core/vtkTestUtilities.h. Post a comment ending in the line Do: reformat to rewrite the MR source branch automatically.
    • commit 76adb175 is not allowed because the following files are not formatted according to the 'clang-format' check: IO/HDF/vtkHDFReader.cxx, IO/HDF/vtkHDFReader.h. Post a comment ending in the line Do: reformat to rewrite the MR source branch automatically.

    Warnings:

    • the merge request is marked as a work-in-progress.

    The warnings do not need to be fixed, but it is recommended to do so.

    Please rewrite commits to fix the errors listed above (adding fixup commits will not resolve the errors) and force-push the branch again to update the merge request.

  • Do: reformat

  • This topic has been reformatted and pushed; please fetch from the source repository and reset your local branch to continue with further development on the reformatted commits.

  • Lucas Givord added 4 commits

    added 4 commits

    • 8d9b07b7 - vtkTestUtilities: support partialy pdc
    • 58d374ae - doc(vtkHDF): add design for composite dataset
    • 8e8cdc69 - vtkHDFReader: add support for composite dataset
    • 3650b7f0 - vtkHDFReader: add release note about composite support

    Compare with previous version

  • Warnings:

    • the merge request is marked as a work-in-progress.

    The warnings do not need to be fixed, but it is recommended to do so.

  • Lucas Givord marked this merge request as ready

    marked this merge request as ready

  • @louis.gombert @keu-sciviz please review

  • @yohann.bearzi could you please take a look at the vtkTestUtilities part ?

  • Nicolas Vuaille
  • Nicolas Vuaille
  • Nicolas Vuaille
  • Lucas Givord added 4 commits

    added 4 commits

    • 62abde85 - vtkTestUtilities: support partialy pdc
    • 93b603d1 - doc(vtkHDF): add design for composite dataset
    • 07044c25 - vtkHDFReader: add support for composite dataset
    • 992f182b - vtkHDFReader: add release note about composite support

    Compare with previous version

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Please register or sign in to reply
    Loading