Skip to content

COMP: Adjust fuzzy compare tolerance

Bill Lorensen requested to merge lorensen/vtk:AdjustFuzzyCompareTolerance into master

Regarding UnitTestFunctionParser...

Dan Lisa reports "We been having failures for this test on amber12 when compiling in Release mode. Debug mode works OK. Intel processors have internal 80 bit floating point registers and can do floating point calculations using those. Using those registers, which happens when enabling optimizations will yield different results than doing computation using 64bit floats."

Inspection of the failures shows that the FuzzyCompare tolerance needs to be adjusted. Using a factor of 128 * epsilon should resolve the failures while maintaining a high level of accuracy for the comparison.

Merge request reports