Skip to content

Fix intermittent UnitTestTimer failures with CUDA

We have been noticing occasional failures of UnitTestTimer on the dashboard for CUDA devices. The timer seems to be recording less time than is actually elapsed.

The problem might be that the CUDA timer actually inserts fences to the CUDA stream rather than record the current time. Thus, the actual time might start after some pending operations complete.

To attempt to match the UnitTestTimer measurements closer to wall-clock time, add a synchronize before starting the timer.

Merge request reports

Loading