BUG: (re-) allocate space for PointIds if vertCount > NumberOfPlanes-1
Using vtkHull
under non-common circumstances (e.g. for Facet analysis as in https://github.com/romangrothausmann/FacetAnalyser/blob/6765680cb4b00fb1107bee890d44b1e2d0a0dad7/code/FacetAnalyser.cxx#L535) can cause malloc(): memory corruption:
or free(): invalid next size (fast):
crashes. It seems these originate from cases where the plane intersections do not form a closed surface. Apart from issuing a Warning/Error of this case vtkHull
should not crash. I do not fully understand the algorithm of vtkHull::ClipPolygonsFromPlanes
but it's logic can increase vertCount
above NumberOfPlanes-1
which then causes a bad reference and assignment at:
romangrothausmann/vtk@ca7bee49
This patch solves this issue by dynamically increasing the size of pnts
to hold the PointIds up to a new vertCount
.
The errors can be reproduced with: https://github.com/romangrothausmann/VTK-CLIs/blob/4d011865bc181f1185150ec0aa855a3e1552a3bd/hull.cxx using these parameters for four planes:
free(): invalid next size (fast)
:
hull hull.vtp 0 0 extent.mha hull.mha 0 6.57508 6.36269 2.49595 0.051209 -0.157065 -0.936725 6.51959 6.34292 2.4979 0.108651 0.435188 -0.809954 6.69346 6.43391 2.61814 0.981994 0.0262945 0.0421564 6.47496 6.38878 2.51752 -0.0467415 -0.432878 0.820604
malloc(): memory corruption
(by inverting second normal):
hull hull.vtp 0 0 extent.mha hull.mha 0 6.57508 6.36269 2.49595 0.051209 -0.157065 -0.936725 6.51959 6.34292 2.4979 -0.108651 -0.435188 0.809954 6.69346 6.43391 2.61814 0.981994 0.0262945 0.0421564 6.47496 6.38878 2.51752 -0.0467415 -0.432878 0.820604