vtk x11 ProcessEvents inconsistent behaviour

When I call ProcessEvents in my own loop, the joystick functionality: Dolly, Spin, Rotate do not work. However, zooming in/out works. This is not the case on windows or SDL2.

void callback(vtkObject* caller, unsigned long eid, void* callData)
{
  ... // cast caller to interactor
  ... // my event loop starts
    interactor->ProcessEvents()
  ... // my event loop ends  
}
interactor->AddObserver(vtkCommand::StartEvent, this, &callback);
interactor->Start()