Unreliable return value in vtkOBBTree::IntersectWithLine(const double p1[3], const double p2[3], vtkPoints *, vtkIdList *)
This issue was created automatically from an original Mantis Issue. Further discussion may take place here.
The return value, which is supposed to tell me if the point p1 is inside or outside, is not reliable. Following addition to the code would make this function much more robust (vtkOBBTree.cxx, around line 888 ):
... delete [] senseList; delete [] cellList; delete [] distanceList; delete [] OBBstack; // return 1 if p1 is inside, 0 is p1 is outside //---- begin added code ---- if(nPoints == 0) rval = 0; else if(nPoints % 2) rval = 1; else rval = -1; //---- end added code ---- return rval; }
For this to work nPoints must be placed outside of the scope of if (listSize != 0) { ... } at line 801.
This change will also make all functions using this function more robust, e.g. vtkOBBTree::InsideOrOutside.
I added a patch (diff oldvtkOBBTree.cxx newOBBTree.cxx).