Skip to content

Fixes and improvements for Streaming mode

Caitlin Ross requested to merge caitlin.ross/fides:streaming-mode-fix into master

Fixes #14 (closed).

The changes in this MR are due to an ADIOS PR from a couple of months ago. In ADIOS, checks were added to the engines when using BeginStep/EndStep that were causing failures in Fides. Most situations we test for were fine, but XGC has some DataSources that contain variables with time steps and some do not. Since DataSetReader manages all DataSources together, it was trying to call BeginStep or EndStep on sources that should not have it called. So DataSource now stores the most recent StepStatus, so once it hits EndOfStream it will no longer call those functions on that source.

The added checks in ADIOS also showed that having both DataSetReader::ReadDataSet() (to be used in random access mode) and DataSetReader::ReadStep() (used in streaming mode, in conjunction with DataSetReader::PrepareNextStep()) is error prone. Since the only difference in these methods is whether they call DoAllReads or EndStep, the reader determines if streaming mode is being used and makes the appropriate call for the mode in ReadDataSet() now. ReadStep() is now deprecated.

Merge request reports