Suppress data redistribution before volume rendering
I would like to ask for advice. I doubt this is an issue per se, but I am struggling with it and couldn't find the details on the Internet. Also, I do not want to contribute to your FAQ on why PV doesn't scale, as I can clearly see that it does, but I am doing something in a bit cumbersome way.
So I am trying to visualise development of thermal stratification layers with volume rendering. I am on a POWER8 system and would like to take advantage of 4 GPU on the node. PV is compiled with EGL as backend. My files are an output of Code_Saturne simulation and are in Ensight which appears to be serial.
What I do is this:
- Open PV with 16 core server and
- Box clip of the data.
- Run D3 on the data. I select "cut cells" for Boundary Mode and 3 layers of ghost cells. I select "minimal memory"
- I save the data to VTM format. I can also set Exodus format. A side question here is what determines the output data format? I noticed that the depending on the set I can save pvtm, vtm, paraview format etc.
- I open another instance of ParaView and render directly from the file collection I generated before.
What I noticed by playing with Timer Log is that PV appears to perform another decomposition just before the rendering:
Regenerate Kd-Tree, 10.6452 seconds Redistributing Data for Ordered Compositing, 71.392 seconds
I've already decomposed the data so can I make PV skip that step?
Generally, I am looking for ways to speed up the interaction and my concern is that redistribution is the main bottleneck. My idea was that explicit decomposing will accelerate the read time from the disk, but my understanding is that I am still reading and redistributing every time the time step changes. Is there any way I can avoid this.
On a similar note is there any way to collate/pipeline the time series read. P8 nodes have sufficient RAM to store my case but I fear I am just hammering the file system unnecessary when animating my volume or contours.
If you have any comments or suggestions, please let me know. Thanks!