ENH: VR support for Vulkan renderer

VR support for Vulkan renderer. This turned out to be a lot of work, but also led to some improvements in the Vulkan renderer and interactor classes.

Status:

• Corrected image layout transitions
• Added additional framebuffer images in (low?) dynamic range to send to the headset
• Headset displays two images now
• Apply HMD transformation to camera
• Refactor interactor classes for Vulkan
• Implement stereo instanced rendering¹
• Update shaders to account to stereo rendering
• Load/render controllers
• Calculate the correct the camera position shader variables for VR mode
• Identify performance bottlenecks and fix
• Clean up destructor for Vulkan renderer
• Fixed bug with uploading static mesh data

Known issues:

• The validation layer is ironically complaining about image layout issues, but I can't identify the images that they affect. This may be internal to OpenVR, and perhaps updating OpenVR can fix these.

1. uses VK_KHR_multiview, which limits support mainly to PC devices. However, this is the most efficient way to render VR currently.

Warnings:

• the merge request is marked as a work-in-progress.

The warnings do not need to be fixed, but it is recommended to do so.

added 1 commit

• 50895f0c - Fixed the linker issue

Compare with previous version

Warnings:

• the merge request is marked as a work-in-progress.

The warnings do not need to be fixed, but it is recommended to do so.

added 1 commit

• 716f1d05 - ENH: added VR support to Vulkan, also corrected image layouts in renderpasses

Compare with previous version

Warnings:

• the merge request is marked as a work-in-progress.

The warnings do not need to be fixed, but it is recommended to do so.

changed the description

added Topic: Rendering Type: new feature labels

added 1 commit

• c586c94f - WIP: added HMD tracking

Compare with previous version

Warnings:

• the merge request is marked as a work-in-progress.

The warnings do not need to be fixed, but it is recommended to do so.

changed the description

marked the checklist item Corrected image layout transitions as completed

marked the checklist item Added additional framebuffer images in (low?) dynamic range to send to the headset as completed

marked the checklist item Headset displays two images now as completed

marked the checklist item Apply HMD transformation to camera as completed

marked the checklist item Apply HMD transformation to camera as incomplete

marked the checklist item Headset displays two images now as incomplete

marked the checklist item Added additional framebuffer images in (low?) dynamic range to send to the headset as incomplete

marked the checklist item Corrected image layout transitions as incomplete

marked the checklist item Corrected image layout transitions as completed

marked the checklist item Added additional framebuffer images in (low?) dynamic range to send to the headset as completed

marked the checklist item Headset displays two images now as completed

marked the checklist item Apply HMD transformation to camera as completed

marked the checklist item Refactor interactor classes for Vulkan as completed

marked the checklist item Implement stereo instanced rendering as completed

marked the checklist item Fix post-processing to account to stereo instanced rendering as completed

added 1 commit

• 54402daf - ENH: added stereo VR support

Compare with previous version

Warnings:

• the merge request is marked as a work-in-progress.

The warnings do not need to be fixed, but it is recommended to do so.

changed the description

changed the description

added 2 commits

• 6b1a6256 - WIP: added controllers
• a239a541 - WIP: merge into VR render commit

Compare with previous version

Warnings:

• the merge request is marked as a work-in-progress.

The warnings do not need to be fixed, but it is recommended to do so.

added 1 commit

• a93d5b07 - WIP: added controller models and fixed performance

Compare with previous version

Warnings:

• the merge request is marked as a work-in-progress.

The warnings do not need to be fixed, but it is recommended to do so.

added 11 commits

• a93d5b07...fe4fdef0 - 3 commits from branch iMSTK:master
• 158f4324 - CLEANUP: moved VR controls to VTK folder
• e14e0f31 - ENH: first VR MR for Vulkan
• 7d4be260 - Fixed the linker issue
• 07fa7b64 - ENH: added VR support to Vulkan, also corrected image layouts in renderpasses
• a1a91da6 - WIP: added HMD tracking
• a3294f6a - ENH: added stereo VR support
• 8752c948 - WIP: added controllers
• fa5f94b5 - WIP: added controller models and fixed performance

Compare with previous version

Warnings:

• the merge request is marked as a work-in-progress.

The warnings do not need to be fixed, but it is recommended to do so.

added 1 commit

• b9120286 - WIP: made some fixes to interactor style

Compare with previous version

Warnings:

• the merge request is marked as a work-in-progress.

The warnings do not need to be fixed, but it is recommended to do so.

changed the description

marked the checklist item Load/render controllers as completed

marked the checklist item Calculate the correct the camera position shader variables for VR mode as completed

marked the checklist item Identify performance bottlenecks and fix as completed

added 1 commit

• 714d0ff1 - WIP: tried to improve bug

Compare with previous version

Warnings:

• the merge request is marked as a work-in-progress.

The warnings do not need to be fixed, but it is recommended to do so.

added 2 commits

• 746c2d34 - ENH: added function to retrieve VR system pointer
• 0d8abc31 - BUG: removed duplicate callback functions

Compare with previous version

Warnings:

• the merge request is marked as a work-in-progress.

The warnings do not need to be fixed, but it is recommended to do so.

@NickMilef will test soon as the HMD is out for a conference!

added 23 commits

• 0d8abc31...b4e91fb0 - 11 commits from branch iMSTK:master
• 5afd536c - CLEANUP: moved VR controls to VTK folder
• 79e8ae2f - ENH: first VR MR for Vulkan
• 46605c34 - Fixed the linker issue
• d31ee881 - ENH: added VR support to Vulkan, also corrected image layouts in renderpasses
• c722ab6b - WIP: added HMD tracking
• a03f1721 - ENH: added stereo VR support
• 6f427a20 - WIP: added controllers
• bbd75a3e - WIP: added controller models and fixed performance
• ed8e6f34 - WIP: made some fixes to interactor style
• 68c3b95d - WIP: tried to improve bug
• 62482c4c - ENH: added function to retrieve VR system pointer
• e91f5604 - BUG: removed duplicate callback functions

Compare with previous version

Warnings:

• the merge request is marked as a work-in-progress.

The warnings do not need to be fixed, but it is recommended to do so.

added 6 commits

• 2679b9ae - ENH: first VR MR for Vulkan
• 1a83d28c - ENH: added VR support to Vulkan, also corrected image layouts in renderpasses
• d6691d93 - ENH: added VR support with HMD tracking
• 3eab2d68 - ENH: added controller support and rendering to Vulkan VR
• c604d320 - ENH: added function to retrieve VR system pointer
• 4801c896 - BUG: removed duplicate callback functions

Compare with previous version