Use header metadata to initialize volume scalar range

Created by: agirault

Co-Authored with @jcfr

Motivation

We're working on a project where we need to speed up the loading of multiple large volumes (Gigabytes) in a custom version of Slicer.

• Our first bottleneck was - FYI - tackled by simply using uncompressed data vs compressed data, where we encountered a 50% to 70% improvement (working with an NVMe SSD where reading from disk is much faster than uncompressing).
• Our second bottleneck was caused by the computation of the volume scalar range, needed to automatically calculate the window/level. The following stack trace obtained during one of our profilings shows a summary of the calls and their cost in CPU time, where GetDisplayScalarRange takes around 40% (in our use case, using uncompressed data, and could go up to 47%. With uncompressed data, the cost was around 17%).

• 97.0% - qSlicerIOManager::loadNodes
  • 86.2% - qSlicerCoreIOManager::loadNodes
    • 82.5% - vtkSlicerVolumesLogic::AddArchetypeVolume
      • 74.9% - vtkMRMLStorageNode::ReadData
        • 44.7% - vtkMRMLVolumeNode::SetAndObserveImageData
          • 43.1% - vtkMRMLScalarVolumeDisplayNode::CalculateAutoLevels
            • 39.2% - vtkMRMLScalarVolumeDisplayNode::GetDisplayScalarRange
              • 39.2% - vtkDataSet::GetScalarRange
            • 3.9% - vtkStreamingDemandDrivenPipeline::Update
              • 3.9%- vtkImageAccumulate::RequestData
        • 29.6% - vtkStreamingDemandDrivenPipeline::Update
          • 26.0% - itk::ImageFileReader::GenerateData()
      • 5.8% - vtkMRMLScene::EndState
    • 2.6% - vtkMRMLApplicationLogic::vtkInternal::PropagateVolumeSelection
  • 10.7% - qProgressDialog::setValue

Objective

Offer a way to use the metadata min/max information from the volume file headers to speed up the data loading if it exists.

Current support

Multiple file formats define properties in the header file to store the scalar range, and ITK adds these info to the metadata dictionary that can be retrieved using itkObject::GetMetaDataDictionary() :

• nifti: [cal_min, cal_max]
  • format standard
  • itk support
• nrrd/nhrd: [min, max]
  • format standard
  • itk support
• mha/mhd : [ElementMin, ElementMax]
  • format standard
  • itk support

Also, the class vtkITKArchetypeImageSeries[Scalar]Reader already retrieves the metadata (1) and sets it to the vtkMRMLVolumeNode (2):

1. https://github.com/Slicer/Slicer/blob/868c60c58d7c4841462fd15cdb178262a645ce44/Libs/vtkITK/vtkITKArchetypeImageSeriesReader.cxx#L816-L826
2. https://github.com/Slicer/Slicer/blob/eefeac9286f48552e8418a11f412b2823a09b407/Libs/MRML/Core/vtkMRMLVolumeArchetypeStorageNode.cxx#L407