ci,docs: backport transtion to webserver for doxygen/sphinx docs

Backporting the following MRs:

• Merge-request: !6963 (merged)
• Merge-request: !6959 (merged)
• Merge-request: !6957 (merged)
• Merge-request: !6954 (merged)
• Merge-request: !6948 (merged)
• Merge-request: !6966 (merged)
• Merge-request: !6969 (merged)