Fixes a shutdown bug in X

When running the visual tests sometimes they would abort with an X error messages, this seems to correct this issue, something similar was reported in vtk/vtk#18242 but the fix wasn't merged (it seems)