Increasing memory usage when reading temporal data
ParaView memory consumption seems increasing when reading temporal data.
The following chart shows the consumption of the physical memory (RSS) with the Xdmf
reader on a ≃ 20GB dataset.
Each red line correspond to a new timestep request:
Same trends are observed with other readers. For example when opening .vtm
files :
Several things to note at this point :
- In each case, things seems to stabilize over time.
- Going back to a previously read timestep or reading a new one has no significative impact on the results.
- Changing the way to display the output from
Render View
toSpreadSheet View
as no significative impact as well. - Using pvpython scripting without rendering involved shows the same results.
One of the possible track to follow was to ensure that the animation cache was used correctly.
When opening a smaller dataset (Xdmf
≃ 3GB) and setting the animation cache on and off, we observe the following results:
We can see that:
- With the animation cache, the memory consumption increases on each timestep. Then by looping the animation, memory consumption doesn't change anymore.
- Without the animation cache, I have the same memory consumption, without the plateau once looping.
The first situation seems coherent, as caching data increases naturally the memory usage; then no more cost when eveything is cached.
But without cache, we can expect no increase of the memory consumption between each timestep. It looks like the data is cached anyway, but not used during the loop.
When trying to reproduce it with VTK alone (with the same input data, Xdmf
≃ 3GB) we can observe similar trend as well :
This may invalidate the fact that the animation cache is the problem.
Now when using no rendering pipeline at all :
Among all the measures I made, this is the only case where memory consumption seems not growing.