Very poor performance when reading large STL files
Paraview's STL reader hangs for without any error messages for a very long time (in excess of 30 minutes) when attempting to read in a binary STL file that is larger than about 500 MB. The STLReader progress bar will reach anywhere from 30 to 80% before the client UI hangs and becomes unresponsive. Tthe STL files are not damaged and the computer has more than enough resources. Tested on Paraview v.5.1.2 64-bit Linux and OS X binaries from the web.
Slow performance with very large files is not unexpected. The strangeness occurs because files about 400 MB in size will load in <30 seconds, whereas a 600 MB file will take 30 to 60 min. There is no swap in use and RAM use is just a few GB. During the long delay, the user has no way to know whether the program is working or has frozen permanently. These large STL files can open in a couple of minutes using other software (i.e. Meshlab, Blender), so the problem is unique to paraview.
On Linux, strace reveals that the File I/O operations have completed and mmap calls are being made. RAM usage is climbing very slowly. The time between mmap calls gets successively slower and slower until they are about 60 seconds apart and the OS reports that the paraview client process has stopped responding. 30 mins later I can see that the paraview process is occasionally making mmap calls, but they are many minutes apart in cpu-time. The paraview client process is at 100% CPU consistently. After about 45 to 60 minutes paraview client suddenly comes back to life and renders the STL file. From that point onward, performance is excellent.
This problem does not occur for STL files that are 500 MB or less in size, these will open after a few minutes. The hang occurs for files roughly above that size. There are no errors reported. This problem may be inside VTKStlReader rather than Paraview, feel free to redirect me to the correct place.
More information can be provided upon request, however obviously it is difficult to upload samples of these STL files as they are extremely large.