Added more flags for using OpenGL libraries to allow combined osmesa (offscreen) + opengl rendering
First attempt (previous branch/merge request) only considered VTK_USE_X and seemed to have broken CDash for MacOSX. This should be fixed now
Merge request reports
Merged by
(Apr 6, 2025 5:41am UTC)
Activity
Testing commands handed to buildbot.
Branch-at: c3d05fab
mentioned in commit 264de7a6
I do not understand why, but it is possible that this commit breaks paraview. I try to MR a VTK update into paraview, containing this commit ( and of course others ), and i have a lot of openGL related errors, while this commit is the only one that has to do with openGL.
see my [WIP] MR : paraview/paraview!753 (closed)
mentioned in merge request paraview/paraview!753 (closed)
I've been bisecting vtk commits, and this is indeed breaking osmesa buildbot on paraview. Check this : https://open.cdash.org/index.php?compare1=63&filtercount=2&field1=buildname%2Fstring&project=ParaView&field2=buildstarttime%2Fdate&showfilters=0&limit=100&compare2=83&value1=842e8098&showfeed=0&value2=20160502T054738
especially these builds :
- blight [linux-shared-release+mpi+opengl2+osmesa+python]
- amber8 [linux-static-release+mpi+osmesa+python]
- amber8 [linux-static-release+osmesa]
The first two just can't find OpenGL. the later can't link to OpenGL.
Looking at the code, it looks like there is a new flag called VTK_USE_MESA. So far, so good, It builds with the flag, but the mesa buildbot would need to update their configuration i supose.
blight is administrated by @utkarsh.ayachit. amber8 by kitware-developers, who is that ?
I do not know about buildbot update process, is there anything automatic or is it done manually ?
Edited by Mathieu Westphal (Kitware)I think the flags have not changed, there is VTK_USE_OSMESA (and not VTK_USE_MESA) which has been there before. The changed behavior is that before, OpenGL "on-screen" rendering could not be combined with offscreen rendering. Now, this is not mutually exclusive anymore.
Before, there was a flag "VTK_CAN_DO_OFFSCREEN" which, as far as I can see, can be replaced with proper values of the other openGL flags, so I guess that indeed this might be fixed by providing a proper configuration to paraview buildbots, e.g. by opening an issue in paraview(?).
Please let me know if my you think that something related to this commit should be changed...
@fabian.wenzel Can you please elaborate on your statement
OpenGL "on-screen" rendering could not be combined with offscreen rendering. Now, this is not mutually exclusive anymore.
When you say it's not mutually exclusive anymore, you mean in VTK or in Mesa3D? To the best of my knowledge you could not link with libGL and libOSMesa since Mesa > 7.9 and hence we stopped supported on-screen and off-screen in the same VTK build.
If you mean that Mesa3D has now changed to allow linking with libGL and libOSMesa, do you have a version number in which this change becomes/became official?
Edited by Utkarsh AyachitWell, I am more a user than a developer of VTK, but this is what I think:
Offscreen rendering, as far as I know, always requires OSMesa on Unix. A couple of years ago it seemed that OSMesa could not easily be combined with On-screen rendering in the sense that names needed to be mangled (there are several posts on the web regarding MangledMesa w.r.t. VTK, I actually never tried that approach and now it seems to be outdated). Later, around VTK 6.10, OSMesa as well as VTK on-screen rendering worked simultaneously on Linux - whereby the platform I have used was headless and probably came with Mesa's implementation of the on-screen OpenGL libraries (Mesa3D?) - version 9.2.3. So at that time it was already possible to combine on- and off-screen rendering.
In VTK 7.0.0, the vtkOpenGL.cmake file (which seemed to be new) introduced a mutually exclusive configuration for On-screen rendering and Off-screen rendering (VTK_USE_X turned off off-screen rendering).
This merge request reintroduced joint usage of on-screen and off-screen capabilities (using - again - Mesa 9.2.3 for both backends).
I would think that combining OSMesa with a hardware-accelerated on-screen OpenGL-driver (e.g. NVidia, Intel etc.) might not be so easy since they do not share code, e.g. the same OpenGL rendering calls would need to find the different backends - which is hard to support... this is why there is a flag that OpenGL can be used together with OSMesa in vtkOpenGL.cmake (which was actually there before, I have just modified the description).
So I do not have a version number after which combining on-screen and off-screen rendering (when Mesa is used for both backends) became official, I just can confirm that it seems to work on my system - the majority of VTK calls require OpenGL libraries, and the SetOffscreenRendering() call additionally requires OSMesa.
mentioned in merge request paraview/paraview!754 (merged)
@fabian.wenzel, Thanks for the info. Alas, I will have to revert this change for now (!1476 (merged)). We don't have a dashboard that tests this configuration hence we cannot verify that it's working. My initial attempts to build Mesa 11.2 (the required version for OSMesa with OpenGL2 backend) with mangling failed to get a working OSMesa build hence I could not verify that this works.
While this is a noble effort and we would like to support this configuration if possible, given it's tendency to disrupt current builds for applications like ParaView, it will need a little more effort including adding dashboard machines to test the new configuration before this gets merged in again.
References for self:
It's been two months, but let me comment today... Actually it's sad to see that this fix did not end up in a release but was reverted. I have just worked on the combination of OpenGL-related CMake variables which were already there before, and which to some extent did not make sense, but apparently just kept an existing CDash configuration for Paraview running.
The approach you, Utkarsh, have done, according to your previous message, is aiming at the OpenGL2 backend, whereas the changes I have made are related (at least are intended) to the OpenGL backend.
Furthermore, since this change is now reverted, it might very well keep your existing CDash configurations for ParaView running, but note that it re-introduced the On-/Off-screen rendering problem on my machine (and, according to the VTK users list, I am not the only one http://vtk.1045678.n5.nabble.com/Re-Combined-on-screen-and-off-screen-possible-with-VTK-7-0-0-on-linux-X-td5736424.html).
I am not attending to blame anyone here, I understand that there are circumstances that lead to the decision to revert this patch. However, I would like to re-introduce the change if possible in the near future and hope that the CDash configurations can be fixed. Unfortunately, when looking at the CDash pages Matthieu has pasted the link for in this thread, I could not deduce the CMake flags in order to verify the problem by inspection.
Is there any way we could do the next step?
@fabian.wenzel totally happy to have this supported again. Like I said earlier, I am indeed interested in supporting this not-currently-supported configuration of using OSMesa and on-screen Mesa in the same build.
It must be noted, however,
- VTK makes no claims in supporting this configuration. It was intentionally dropped since Mesa stopped supporting it with 7.9 through 9.2 (as you say). Thus, this is now a new configuration and of course, we'd need to update our testbed to support it.
- Unless I am mistaken, there has been no release of VTK for past 5 months. So even if this wasn't reverted it would not have been in any release.
Since you have had success with this, can you provide information on building the latest Mesa with offscreen and onscreen configurations correctly? Then I can have a dashboard setup to test that.
Edited by Utkarsh Ayachit@utkarsh.ayachit Fine, let's do it. I have just installed a fresh Ubuntu 16.04 on my VM and compiled VTK 7.0.0 (with the patch of this merge request) successfully. Both, on-screen and off-screen rendering seem to work. I did not have to do any tweaking of Mesa. The ubuntu packages (Mesa-X / OSMesa 11.2.0 - I hope this is recent enough) seem to work out-of-the box and hand in hand. In fact, when doing ldd /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1 and ldd /usr/lib/x86_64-linux-gnu/libOSMesa.so.8, I see that both depend on the same /usr/lib/x86_64-linux-gnu/libglapi.so.0. Actually, I have not found information about Mesa not supporting combined on-screen and off-screen rendering, maybe I have not used the correct search terms?
How did I test?
I have checked on-screen rendering by running "python Examples/Tutorial/Step1/Python/Cone.py": I see a window and the rotating cone.
I have checked off-screen rendering by adding "renWin.SetOffScreenRendering(1)" to the same script when setting up the render window and replacing the loop for rotating the cone with a screenshot python code snippet (windowToImageFilter, PNGWriter). When running, there is no window popping up and the screenshot correctly shows the cone.
The configuration I have used contains the following flags:
cmake -D VTK_RENDERING_BACKEND="OpenGL" -D VTK_USE_OSMESA=True -D VTK_WRAP_PYTHON=True -D VTK_PYTHON_VERSION=3
@fabian.wenzel Mind testing this change !1642 (merged). It's in the same spirit as your changes, except it keeps
VTK_USE_OSMESAinternal.Thanks, it seems to work, I have voted on change !1642 (merged).
Thanks for reviewing. I'll have !1642 (merged) merged once a few folks get a chance to give it a look.
Mentioned in commit 264de7a6
