MemoryError while reading and displaying data from csv on an imported mesh using programmable filter
Hi Team, We have used paraview's programmable filter to automate the reading of csv data (Columns with different variables and their values)and displaying the contours of these variables in paraview on the imported mesh. The rows of csv equal to the number of nodes of the mesh. We have 5 different csv files for 5 different time steps. Following programmable filter (in Request Information tab) first resets the timesteps to the required values:
frequency = [100.0, 200.0, 300.0, 400.0, 500.0] def SetOutputTimesteps(algorithm, timesteps): executive = algorithm.GetExecutive() outInfo = executive.GetOutputInformation(0) outInfo.Remove(executive.TIME_STEPS()) for timestep in timesteps: outInfo.Append(executive.TIME_STEPS(),timestep) outInfo.Remove(executive.TIME_RANGE()) outInfo.Append(executive.TIME_RANGE(),timesteps) outInfo.Append(executive.TIME_RANGE(),timesteps[-1]) SetOutputTimesteps(self,(frequency))
After setting the timesteps following programmable filter (in Script tab) is used to read the csv file corresponding the the current chosen timestep and append data of each columns along with the variable name to the mesh in paraview:
import numpy as np import csv def GetUpdateTimestep(algorithm): executive = algorithm.GetExecutive() outInfo = executive.GetOutputInformation(0) if not outInfo.Has(executive.UPDATE_TIME_STEP()): return None return outInfo.Get(executive.UPDATE_TIME_STEP()) curr_freq = GetUpdateTimestep(self) f = open(str(curr_freq)+'.csv','r') f_csv = csv.reader(f, delimiter=',') a = 0 A =  for row in f_csv: if(a==0): a += 1 for column in row: A.append(column) globals()[column] =  continue if (a>0): i = 0 for column in row: globals()[A[i]].append(column) i += 1 for ele in A: globals()[ele] = globals()[ele][1:] globals()[ele+'_np'] = np.array(globals()[ele]) globals()[ele+'_np'] = globals()[ele+'_np'].astype(np.float) output.PointData.append(globals()[ele+'_np'], ele) output.GetInformation().Set(output.DATA_TIME_STEP(), curr_freq)
These codes were working initially in Paraview 5.3 on a Windows 7 desktop (4Gb RAM) around 8 months ago, but now same codes and csv data are giving a memory error when we are trying to run them in Paraview 5.3 on both Linux and Windows 10 environments. We are getting the following error:
File "D:\ParaView 5.3.0-Qt5-OpenGL2-Windows-64bit\bin\lib\site-packages\vtk\numpy_interface\dataset_adapter.py", line 676, in append narray = narray.flatten() * numpy.ones((arrLength, components)) MemoryError
If we are reducing the rows in the csv files to 10 rows then the same code works fine. We would be grateful if you kindly help us in this issue. Thanks and Regards, Prateek.