Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • VTK VTK
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 829
    • Issues 829
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 239
    • Merge requests 239
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • VTKVTK
  • VTKVTK
  • Issues
  • #17997
Closed
Open
Issue created Sep 03, 2020 by Ming Jin@mjin_origen

Unable to allocate memory in vtkGenericDataArray.txx when loading large .vtp file, after compiling with Emscripten

Hi,

This is sort of a follow-up question to Issue #17995 (closed) (also relevant: https://github.com/Kitware/vtk-js/issues/1562).
I have a large .vtp file that I'm trying to load in a frontend with javascript, and I am trying to use VTK/C++ as a WebAssembly component.

Here's some example code that I have been experimenting with to load and render the .vtp file using WebGL:

#include "vtkXMLPolyDataReader.h"
#include "vtkSmartPointer.h"
#include "vtkPolyDataMapper.h"
#include "vtkActor.h"
#include "vtkRenderer.h"
#include "vtkSDL2OpenGLRenderWindow.h"
#include "vtkSDL2RenderWindowInteractor.h"

int main ( int argc, char *argv[] )
{
  using namespace std;
  string filename = "./example.vtp";

  // Read all the data from the file
  vtkSmartPointer<vtkXMLPolyDataReader> reader =
    vtkSmartPointer<vtkXMLPolyDataReader>::New();
  reader->SetFileName(filename.c_str());
  reader->Update();
  
  // Visualize
  vtkSmartPointer<vtkPolyDataMapper> mapper =
    vtkSmartPointer<vtkPolyDataMapper>::New();
  mapper->SetInputConnection(reader->GetOutputPort());

  vtkSmartPointer<vtkActor> actor =
    vtkSmartPointer<vtkActor>::New();
  actor->SetMapper(mapper);

  vtkSmartPointer<vtkRenderer> renderer =
    vtkSmartPointer<vtkRenderer>::New();
  vtkSmartPointer<vtkSDL2OpenGLRenderWindow> renderWindow =
    vtkSmartPointer<vtkSDL2OpenGLRenderWindow>::New();
  renderWindow->AddRenderer(renderer);
  vtkSmartPointer<vtkSDL2RenderWindowInteractor> renderWindowInteractor =
    vtkSmartPointer<vtkSDL2RenderWindowInteractor>::New();
  renderWindowInteractor->SetRenderWindow(renderWindow);

  renderer->AddActor(actor);
  renderer->SetBackground(.3, .6, .3); // Background color green

  renderWindow->Render();
  renderWindowInteractor->Start();

  return EXIT_SUCCESS;
}

I have managed to package/preload example.vtp (~6 million grids, over 500MB), and am getting the following errors (on Firefox 80.0:

ERROR: In /work/src/Common/Core/vtkGenericDataArray.txx, line 389 ReadPolyData.js:2270:16
vtkFloatArray (0x58b768): Unable to allocate 155551200 elements of size 4 bytes. ReadPolyData.js:2270:16
<empty string> ReadPolyData.js:2270:16
exception thrown: 5814304 ReadPolyData.js:13325:10
Uncaught 5814304 ReadPolyData.js:13329:5

On Chrome, it's slightly different:

ReadPolyData.js:2270 ERROR: In /work/src/Common/Core/vtkGenericDataArray.txx, line 389
put_char @ ReadPolyData.js:2270
ReadPolyData.js:2270 vtkLongLongArray (0x58c728): Unable to allocate 155551200 elements of size 8 bytes. 
put_char @ ReadPolyData.js:2270
ReadPolyData.js:2270 
put_char @ ReadPolyData.js:2270
ReadPolyData.js:13325 exception thrown: 5819304
callMain @ ReadPolyData.js:13325
ReadPolyData.js:223 Uncaught 5819304

And the error is at this line in the C++ code:

reader->Update();

It appears that the error is not on the WebGL part yet, and already vtkXMLPolyDataReader is having trouble loading the poly data.

I tried to dig through the source code in vtkXMLPolyDataReader. As I am not very familiar with VTK, especially for IO/model loading, I can't quite figure out this error. Could you shed some light on how the IO works here? How does vtkAlgorithm::Update() lead to the conversion into vtkDataArray types? What is the workflow like?

Thanks!

Edited Sep 03, 2020 by Ming Jin
Assignee
Assign to
Time tracking