PSTLWriter hangs when using 3 or more MPI processors
To replicate, do the following:
- Linux, 5.10.0-RC1, remote server. Use 3 or more ranks.
- Load g1s1-10fps.e.16.00. Apply.
- Extract Surface. Apply.
- File/ Save Data. server hangs
If you cancel the server while it's hanging with ctrl+c
, you get the following output
Stack trace:
38 0x55d20ba3550e ./pvserver(+0x250e) [0x55d20ba3550e]
37 0x7f3987061b25 __libc_start_main + 213
36 0x55d20ba35490 ./pvserver(+0x2490) [0x55d20ba35490]
35 0x7f39853a2e3f vtkMultiProcessController::BroadcastProcessRMIs(int, int) + 239
34 0x7f39853a2bf8 vtkMultiProcessController::ProcessRMI(int, void*, int, int) + 232
33 0x7f3985a5c4f5 vtkPVSessionCore::ExecuteStreamSatelliteCallback() + 181
32 0x7f3985a5c297 vtkPVSessionCore::ExecuteStreamInternal(vtkClientServerStream const&, bool) + 247
31 0x7f39855b103d vtkClientServerInterpreter::ProcessStream(vtkClientServerStream const&) + 29
30 0x7f39855b0b4e vtkClientServerInterpreter::ProcessOneMessage(vtkClientServerStream const&, int) + 190
29 0x7f39855b0a2d vtkClientServerInterpreter::ProcessCommandInvoke(vtkClientServerStream const&, int) + 1229
28 0x7f3985f653f3 vtkParallelSerialWriterCommand(vtkClientServerInterpreter*, vtkObjectBase*, char const*, vtkClientServerStream const&, vtkClientServerStream&, void*) + 3267
27 0x7f398349376f vtkParallelSerialWriter::Write() + 63
26 0x7f39852ff8d7 vtkStreamingDemandDrivenPipeline::Update(int, vtkInformationVector*) + 279
25 0x7f39852fe509 vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 921
24 0x7f39852a3a9d vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 877
23 0x7f398529de21 vtkCompositeDataPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 129
22 0x7f39852a11a9 vtkDemandDrivenPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 57
21 0x7f39852a6c40 vtkExecutive::CallAlgorithm(vtkInformation*, int, vtkInformationVector**, vtkInformationVector*) + 80
20 0x7f39834939f4 vtkParallelSerialWriter::RequestData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 420
19 0x7f3983493380 vtkParallelSerialWriter::WriteATimestep(vtkDataObject*) + 1728
18 0x7f3983492665 vtkParallelSerialWriter::WriteAFile(std::string const&, vtkDataObject*) + 501
17 0x7f39852ff8d7 vtkStreamingDemandDrivenPipeline::Update(int, vtkInformationVector*) + 279
16 0x7f39852fe509 vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 921
15 0x7f39852a3a9d vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 877
14 0x7f398529de21 vtkCompositeDataPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 129
13 0x7f39852a11a9 vtkDemandDrivenPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 57
12 0x7f39852a6c40 vtkExecutive::CallAlgorithm(vtkInformation*, int, vtkInformationVector**, vtkInformationVector*) + 80
11 0x7f39830285d7 vtkReductionFilter::RequestData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) + 103
10 0x7f3983027e2b vtkReductionFilter::Reduce(vtkDataObject*, vtkDataObject*) + 1163
9 0x7f398539863c vtkCommunicator::Gather(vtkDataObject*, std::vector<vtkSmartPointer<vtkDataObject>, std::allocator<vtkSmartPointer<vtkDataObject> > >&, int) + 268
8 0x7f3985397d27 vtkCommunicator::GatherV(vtkDataArray*, vtkDataArray*, vtkSmartPointer<vtkDataArray>*, int) + 103
7 0x7f3985397ba1 vtkCommunicator::GatherV(vtkDataArray*, vtkDataArray*, vtkIdTypeArray*, vtkIdTypeArray*, int) + 433
6 0x7f398539789b vtkCommunicator::GatherV(vtkDataArray*, vtkDataArray*, long long*, long long*, int) + 235
5 0x7f39800aeb7d vtkMPICommunicator::GatherVVoidArray(void const*, void*, long long, long long*, long long*, int, int) + 253
4 0x7f397ffeb010 PMPI_Gatherv + 832
3 0x7f397c128696 mca_pml_ob1_send + 822
2 0x7f397dd2bf5c opal_progress + 44
1 0x7f397c0d2bcc ompi_coll_libnbc_progress + 12
0 0x7f3987076da0 /usr/lib/libc.so.6(
This issue was detected when I was trying to solve #19144 (closed).
It might be a result of !3351 (merged) (to be checked).