Speed up sideset reads in the IOSS reader/library reading Exodus
We found a major performance degredation in the IOSS reader/library. This was mostly solved in #22227 (closed), by changing
dbase->set_surface_split_type(Ioss::SPLIT_BY_TOPOLOGIES);
to dbase->set_surface_split_type(Ioss::SPLIT_BY_ELEMENT_BLOCKS);
This changed the performance degredation from about 2 orders of magnitude slower to 4x slower. We still want to discuss, research and possibly implement speeding current code up to the speeds of the Exodus reader/library.
Here is a comment from Greg Sjaardema from #22227 (closed):
OK, I think the issue is that the caching that IOSS does is to check whether the current side's parent element is from the same element block as the previous side's parent element. The sidesets in this model somewhat follow that assumption, but do switch back and forth a few/many times which results in a new query of the element block connectivity.
So, we aren't seeing 199,427 exodus calls, but we aren't seeing 2 either...
There are a few ways to fix this. I need to see what would be the least intrusive and behavior changing...
- Can sort the elements (based on internal id, not global id) in the SideBlock lists which would guarantee an orderly progression through the element blocks.
- Could process the sideblock element lists multiple times -- once per block. Fairly easy, but need to calculate some offsets based on the face node counts to know where to store the connectivity when skpping over elements that are not in the current element block. Not too difficult, but some added bookkeeping.
- Cache element connectivity for multiple element blocks instead of just the "current" block. Uses more memory.
- If sidesets use the SPLIT_BY_ELEMEMT_BLOCK method for generating homogenous sideblocks, then process in element block order by default.