Missing files on LANL machines when writing in parallel
On a particular LANL machine, when writing vtm files in parallel, sometimes some ranks will encounter an error, and will not end up writing files. This seems to occur at higher rank count, something like ~50 ranks, and happens at random timesteps. Currently investigating what the exact conditions are for this error to occur. Modifications may be needed for the XML parallel writers.
` ^[[0m^[[31m(1546.502s) [pvbatch.423 ]vtkXMLCompositeDataWrit:374 ERR|
vtkXMLPMultiBlockDataWriter (0x5b11a10): Sorry unable to create directory: pv_files/grid_005649
Last system error was: No such file or directory^[[0m ^[[0m^[[31m(1546.502s) \
[pvbatch.431 ] vtkXMLWriter.cxx:649 ERR| vtkXMLUnstructuredGridWriter (0x66a0ad0): Error opening output file "pv_files/grid_005649/grid_005649_0_431.vtu"^[[0m ^[[0m^[[31m(1546.502s) \
[pvbatch.431 ] vtkXMLWriter.cxx:652 ERR| vtkXMLUnstructuredGridWriter (0x66a0ad0): Error code "No such file or directory"^[[0m ^[[0m^[[31m(1546.502s) \
[pvbatch.431 ] vtkExecutive.cxx:753 ERR| vtkCompositeDataPipeline (0x62ff990): Algorithm vtkXMLUnstructuredGridWriter(0x66a0ad0) returned failure for request: vtkInformation (0x630d040) Debug: Off Modified Time: 414228 Reference Count: 1 Registered Events: (none) Request: REQUEST_DATA FROM_OUTPUT_PORT: -1 ALGORITHM_AFTER_FORWARD: 1 FORWARD_DIRECTION: 0 `