The vtkCocoaRenderWindow DPI isn't set before first render
Proper text rendering requires that the DPI of the render window is doubled for HiDPI/Retina displays. The vtkCocoaGLView handles this for viewDidChangeBackingProperties and viewWillMoveToWindow notifications. However, these do not guarantee that the DPI is correctly set before the first render. As a result, on Retina displays, the text size is often incorrect (half-size) at first.
One tricky problem is that VTK testing requires the current behavior. The tests use vtkRenderWindow::SetSize(), which sets the window size in device pixels (i.e. half-size pixels). So if the DPI is doubled for Retina, the text will be double-size relative to the image and the regression image will not match. TLDR: The tests always expect 72 DPI (the default) and fail if the DPI is doubled to 144.
A potential compromise is to make automatic DPI adjustment optional. This would allow 72 DPI to be used for the testing, and scaled DPI to be used for applications.