Crash attempting iso-contour on certain AMReX datasets
Hello,
We are experiencing an error/crash when trying to use the iso-contour filter for some datasets generated by AMR-wind. An example of the output is (hopefully) attached here. We've done a bit of testing to narrow things down a bit:
- Crash happens when trying to use certain isovalues on certain quantities (like .1 for q-criterion). Seems to be the more geometry generated, the greater chance of crash.
- Crash happens when using PVserver, regardless the number of MPI ranks ( >= 1)
- Does not happen with GUI-mode Paraview with no MPI
- All output files can be read by other tools (like the python YT package) parallel or not, and at all isovalues
- Seems to happen on our supported versions of Paraview; 5.9, 5.10.
- Doesn’t seem to be an out-of-memory issue. Memory usage seems to be just fine and below machine limits. Tried with a lot more MPI ranks across more nodes to be sure; still crashes.
- Seems to start happening from one tilmestep to the next; beginning sets iso-contour just fine, then the next (and all subsequent time steps) will end in a crash. Lends a bit more credence in some kind of geometry issue.
- Nothing significantly different between a working and non-working plt file on-disk structure
The filter chain we use for this case is very simple:
- Pltfile with just one quantity (q-criterion)
- Cell Data to Point Data filter
- Contour
When using these filters at a problematic isovalue, we get a bit of loading, and then the error log fills up; this is then followed by pvserver stalling out.
We're happy to provide an example dataset, though it is several hundred gigabytes in size, so there might be some difficulties there. Happy to provide more details, or run more tests to help figure this out.