Skip to content
GitLab
Closed
Issue created by Shuhao Wu@shuhaowuContributor

Find Data breaks when running with mpiexec pvserver

I'm currently trying to find a node where U == max(U) using the Find Data dialog. Paraview is connected to a pvserver instance running with np=4. The source filter has 3388 cells and 1794 nodes and is a polygonal mesh (comes from a slice of a structured grid).

Once I hit Run Selection Query, the UI freezes and the server emits the following error:

Error: Failed to evaluate Expression 'U  == max(U)'. The following exception stack should provide additional developer specific information. This typically implies a malfor
med expression. Verify that the expression is valid.                                                                                                        
                                   
(  38.351s) [pvserver.0      ]  vtkPythonSelector.cxx:76    WARN| vtkPythonSelector (0x88ca890): Could not invoke 'python_selector.execute()'              
Traceback (most recent call last):                    
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/paraview/detail/python_selector.py", line 79, in execute
Error: Failed to evaluate Expression 'U  == max(U)'. The following exception stack should provide additional developer specific information. This typically implies a malfor
med expression. Verify that the expression is valid.         
                                                                                                    
(  38.351s) [pvserver.3      ]  vtkPythonSelector.cxx:76    WARN| vtkPythonSelector (0x51a89c0): Could not invoke 'python_selector.execute()'
Traceback (most recent call last):                                                                                                                         
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/paraview/detail/python_selector.py", line 79, in execute
    maskArray = calculator.compute(inputs, query, ns=elocals)                                                                                  
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/paraview/detail/calculator.py", line 143, in compute
    maskArray = calculator.compute(inputs, query, ns=elocals)                   
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/paraview/detail/calculator.py", line 143, in compute
    retVal = eval(expression, globals(), mylocals)                                                                                                                       
  File "<string>", line 1, in <module> 
      File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/vtkmodules/numpy_interface/algorithms.py", line 361, in max     
retVal = eval(expression, globals(), mylocals)
  File "<string>", line 1, in <module>
Error: Failed to evaluate Expression 'U  == max(U)'. The following exception stack should provide additional developer specific information. This typically implies[40/1921]
med expression. Verify that the expression is valid.
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/vtkmodules/numpy_interface/algorithms.py", line 361, in max

    return _global_func(MaxImpl(), array, axis, controller)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/vtkmodules/numpy_interface/algorithms.py", line 188, in _global_func
    return _global_func(MaxImpl(), array, axis, controller)
    res = impl.op()(array, axis)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/vtkmodules/numpy_interface/algorithms.py", line 188, in _global_func
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/vtkmodules/numpy_interface/internal_algorithms.py", line 373, in max
(  38.351s) [pvserver.2      ]  vtkPythonSelector.cxx:76    WARN| vtkPythonSelector (0x41a04c0): Could not invoke 'python_selector.execute()'
    res = impl.op()(array, axis)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/vtkmodules/numpy_interface/internal_algorithms.py", line 373, in max
    ans = numpy.max(narray, axis)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 2505, in amax
Traceback (most recent call last):
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/paraview/detail/python_selector.py", line 79, in execute
    ans = numpy.max(narray, axis)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 2505, in amax
    maskArray = calculator.compute(inputs, query, ns=elocals)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/paraview/detail/calculator.py", line 143, in compute
    retVal = eval(expression, globals(), mylocals)
  File "<string>", line 1, in <module>
    initial=initial)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 84, in _wrapreduction
    return reduction(axis=axis, out=out, **passkwargs)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/numpy/core/_methods.py", line 28, in _amax
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/vtkmodules/numpy_interface/algorithms.py", line 361, in max
    return umr_maximum(a, axis, None, out, keepdims, initial)
    initial=initial)ValueError: zero-size array to reduction operation maximum which has no identity

  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 84, in _wrapreduction
    return reduction(axis=axis, out=out, **passkwargs)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/numpy/core/_methods.py", line 28, in _amax
    return umr_maximum(a, axis, None, out, keepdims, initial)
ValueError: zero-size array to reduction operation maximum which has no identity
    return _global_func(MaxImpl(), array, axis, controller)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/vtkmodules/numpy_interface/algorithms.py", line 188, in _global_func
    res = impl.op()(array, axis)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/vtkmodules/numpy_interface/internal_algorithms.py", line 373, in max
    ans = numpy.max(narray, axis)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 2505, in amax
Error: Failed to evaluate Expression 'U  == max(U)'. The following exception stack should provide additional developer specific information. This typically implies a malfor
med expression. Verify that the expression is valid.
(  38.353s) [pvserver.1      ]  vtkPythonSelector.cxx:76    WARN| vtkPythonSelector (0x81c39d0): Could not invoke 'python_selector.execute()'
Traceback (most recent call last):
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/paraview/detail/python_selector.py", line 79, in execute
    maskArray = calculator.compute(inputs, query, ns=elocals)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/paraview/detail/calculator.py", line 143, in compute
    initial=initial)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 84, in _wrapreduction
        return reduction(axis=axis, out=out, **passkwargs)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/numpy/core/_methods.py", line 28, in _amax
retVal = eval(expression, globals(), mylocals)
  File "<string>", line 1, in <module>
    return umr_maximum(a, axis, None, out, keepdims, initial)
ValueError: zero-size array to reduction operation maximum which has no identity
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/vtkmodules/numpy_interface/algorithms.py", line 361, in max
    return _global_func(MaxImpl(), array, axis, controller)
  File "/home/shuhao/ParaView-5.7.0-RC4-osmesa-MPI-Linux-Python3.7-64bit/lib/python3.7/site-packages/vtkmodules/numpy_interface/algorithms.py", line 217, in _global_func
    comm.Allreduce([res, mpi_type], [res_recv, mpi_type], impl.mpi_op())
  File "mpi4py/MPI/Comm.pyx", line 714, in mpi4py.MPI.Comm.Allreduce
mpi4py.MPI.Exception: Message truncated, error stack:
PMPI_Allreduce(450)...............: MPI_Allreduce(sbuf=0x7fb35c8bd300, rbuf=0x4bf1c80, count=1, datatype=MPI_DOUBLE, op=MPI_MAX, comm=MPI_COMM_WORLD) failed
PMPI_Allreduce(436)...............:
MPIR_Allreduce_impl(293)..........:
MPIR_Allreduce_intra_auto(178)....:
MPIR_Allreduce_intra_auto(84).....:
MPIR_Bcast_impl(310)..............:
MPIR_Bcast_intra_auto(223)........:
MPIR_Bcast_intra_binomial(112)....:
MPIDI_CH3U_Receive_data_found(131): Message from rank 0 and ta

The only way to get out of the frozen UI on the client is to either force quit or shutdown the pvserver.

I can confirm this bug doesn't exist when the pvserver is not running under mpiexec and thus using a single core or when the same operation is performed on the builtin paraview instance, hinting at a parallelization bug.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information