Skip to content

BUG: (re-) allocate space for PointIds if vertCount > NumberOfPlanes-1

Roman Grothausmann requested to merge romangrothausmann/vtk:DEBUGvtkHull into master

Using vtkHull under non-common circumstances (e.g. for Facet analysis as in https://github.com/romangrothausmann/FacetAnalyser/blob/6765680cb4b00fb1107bee890d44b1e2d0a0dad7/code/FacetAnalyser.cxx#L535) can cause malloc(): memory corruption: or free(): invalid next size (fast): crashes. It seems these originate from cases where the plane intersections do not form a closed surface. Apart from issuing a Warning/Error of this case vtkHull should not crash. I do not fully understand the algorithm of vtkHull::ClipPolygonsFromPlanes but it's logic can increase vertCount above NumberOfPlanes-1 which then causes a bad reference and assignment at: romangrothausmann/vtk@ca7bee49

This patch solves this issue by dynamically increasing the size of pnts to hold the PointIds up to a new vertCount.

The errors can be reproduced with: https://github.com/romangrothausmann/VTK-CLIs/blob/4d011865bc181f1185150ec0aa855a3e1552a3bd/hull.cxx using these parameters for four planes:

free(): invalid next size (fast):

hull hull.vtp 0 0 extent.mha hull.mha 0 6.57508 6.36269 2.49595 0.051209 -0.157065 -0.936725 6.51959 6.34292 2.4979 0.108651 0.435188 -0.809954 6.69346 6.43391 2.61814 0.981994 0.0262945 0.0421564 6.47496 6.38878 2.51752 -0.0467415 -0.432878 0.820604

malloc(): memory corruption (by inverting second normal):

hull hull.vtp 0 0 extent.mha hull.mha 0 6.57508 6.36269 2.49595 0.051209 -0.157065 -0.936725 6.51959 6.34292 2.4979 -0.108651 -0.435188 0.809954 6.69346 6.43391 2.61814 0.981994 0.0262945 0.0421564 6.47496 6.38878 2.51752 -0.0467415 -0.432878 0.820604

Edited by Roman Grothausmann

Merge request reports