Skip to content

Fixes to vtkMath::Cross() and vtkPolygon::ComputeNormal()

David Gobbi requested to merge dgobbi/vtk:cross-product-fixes into master

This fixes two cross-product problems introduced by VTK 9.3.

The first problem was overflow and underflow in vtkPolygon::ComputeNormal() for very large and very small polygons. Polygons with edge length of 1e20 or greater would give a normal of NAN, and polygons with an edge length of 1e-23 would give a normal of (0,0,0). These polygons worked with VTK 9.2 and earlier. Using double-precision for the cross product fixes the problem.

The second problem, which was introduced at the same time, is #19365 (closed). The fix adds temporary storage for the cross product so that the same memory can be used for the input and output.

Closes: #19365 (closed)
Backport: release

Edited by David Gobbi

Merge request reports