Unable to allocate memory in vtkGenericDataArray.txx when loading large .vtp file, after compiling with Emscripten
Hi,
This is sort of a follow-up question to Issue #17995 (closed) (also relevant: https://github.com/Kitware/vtk-js/issues/1562).
I have a large .vtp file that I'm trying to load in a frontend with javascript, and I am trying to use VTK/C++ as a WebAssembly component.
Here's some example code that I have been experimenting with to load and render the .vtp file using WebGL:
#include "vtkXMLPolyDataReader.h"
#include "vtkSmartPointer.h"
#include "vtkPolyDataMapper.h"
#include "vtkActor.h"
#include "vtkRenderer.h"
#include "vtkSDL2OpenGLRenderWindow.h"
#include "vtkSDL2RenderWindowInteractor.h"
int main ( int argc, char *argv[] )
{
using namespace std;
string filename = "./example.vtp";
// Read all the data from the file
vtkSmartPointer<vtkXMLPolyDataReader> reader =
vtkSmartPointer<vtkXMLPolyDataReader>::New();
reader->SetFileName(filename.c_str());
reader->Update();
// Visualize
vtkSmartPointer<vtkPolyDataMapper> mapper =
vtkSmartPointer<vtkPolyDataMapper>::New();
mapper->SetInputConnection(reader->GetOutputPort());
vtkSmartPointer<vtkActor> actor =
vtkSmartPointer<vtkActor>::New();
actor->SetMapper(mapper);
vtkSmartPointer<vtkRenderer> renderer =
vtkSmartPointer<vtkRenderer>::New();
vtkSmartPointer<vtkSDL2OpenGLRenderWindow> renderWindow =
vtkSmartPointer<vtkSDL2OpenGLRenderWindow>::New();
renderWindow->AddRenderer(renderer);
vtkSmartPointer<vtkSDL2RenderWindowInteractor> renderWindowInteractor =
vtkSmartPointer<vtkSDL2RenderWindowInteractor>::New();
renderWindowInteractor->SetRenderWindow(renderWindow);
renderer->AddActor(actor);
renderer->SetBackground(.3, .6, .3); // Background color green
renderWindow->Render();
renderWindowInteractor->Start();
return EXIT_SUCCESS;
}
I have managed to package/preload example.vtp (~6 million grids, over 500MB), and am getting the following errors (on Firefox 80.0:
ERROR: In /work/src/Common/Core/vtkGenericDataArray.txx, line 389 ReadPolyData.js:2270:16
vtkFloatArray (0x58b768): Unable to allocate 155551200 elements of size 4 bytes. ReadPolyData.js:2270:16
<empty string> ReadPolyData.js:2270:16
exception thrown: 5814304 ReadPolyData.js:13325:10
Uncaught 5814304 ReadPolyData.js:13329:5
On Chrome, it's slightly different:
ReadPolyData.js:2270 ERROR: In /work/src/Common/Core/vtkGenericDataArray.txx, line 389
put_char @ ReadPolyData.js:2270
ReadPolyData.js:2270 vtkLongLongArray (0x58c728): Unable to allocate 155551200 elements of size 8 bytes.
put_char @ ReadPolyData.js:2270
ReadPolyData.js:2270
put_char @ ReadPolyData.js:2270
ReadPolyData.js:13325 exception thrown: 5819304
callMain @ ReadPolyData.js:13325
ReadPolyData.js:223 Uncaught 5819304
And the error is at this line in the C++ code:
reader->Update();
It appears that the error is not on the WebGL part yet, and already vtkXMLPolyDataReader is having trouble loading the poly data.
I tried to dig through the source code in vtkXMLPolyDataReader. As I am not very familiar with VTK, especially for IO/model loading, I can't quite figure out this error. Could you shed some light on how the IO works here? How does vtkAlgorithm::Update()
lead to the conversion into vtkDataArray types? What is the workflow like?
Thanks!