ImGui integration?

In this case, necessity would drive development and support for DearImGUI.

Is ImGui planned to be integrated (as Qt is) on the short or long term?

No plans afaik, not unless we've got an external customer who can fund that work.

We've not yet seen a dire need for DearImGUI other than the cool factor. I took a stab at it almost 2 years ago in vtkDearImGUIInjector and the live web demo. It started off as a fork of trlsmax/imgui-vtk and then changed direction to inject ImGUI into a VTK app rather than showing VTK inside a DearImGUI graphics app.

What I like about the immediate mode GUIs is that there is no state duplication at all. Also, it allows the exact look across desktop and web applications - through WebAssembly far easier than compiling Qt to wasm.

At the end of the day, I'd probably use DearImGUI as a developer UI rather than a user-facing UI, but the gallery in DearImGUI shows some eye candy user-facing UIs too, so it seems that the sky is the limit :)

In this case, necessity would drive development and support for DearImGUI.

100% agree! Can't answer the question, but, ImGui seems adopted more and more in more and more places... Except VTK?

We've not yet seen a dire need for DearImGUI other than the cool factor.

100% understand that! Cool factor is an important one

It started off as a fork of trlsmax/imgui-vtk and then changed direction to inject ImGUI into a VTK app rather than showing VTK inside a DearImGUI graphics app.

Looking at theses codes theses days (before opening this issue) led me to the same conclusion, and, it would have been probably better to have moved the opposite direction (= integrating ImGui in VTK).

At the time (I may be wrong), my understanding is that creating a vtkExternalOpenGLRenderWindowInteractor (to be used with vtkGenericOpenGLRenderWindow) that wraps ImGui would be enough to get ImGui integrated into VTK. Not sure I can acheive this, and if so, not sure it'd be worth PR'ing this here.

What I like about the immediate mode GUIs is that there is no state duplication at all.

100% agree! :) And unlike Qt, you master what do your threads and how (in Qt, I always have some kind of impression that you use something but don't really know how is working inside.... Which make sense when you want to use a library to abstract things you don't want to deal with, but, make things difficult to dev / debug when you have a specific off-road use-case)

external customer who can fund that work.

I guess there is no demand because it's not here, and that, as soon as there would be even a beginning of it, demands will pop up! :)

@michael.migliore

No plans afaik, not unless we've got an external customer who can fund that work.

Or a code contribution from a VTK community member.

Not sure I can acheive this

Or a code contribution from a VTK community member.

Not sure to understand steps to follow to get ImGui integration (based on GLFW + GLAD): if somebody can help describing what is supposed to be done why and how, would be helpful! :)

So, DearImGUI integration is broken down to two parts. First step is getting the UI to render along with VTK's frames. The second step is passing through VTK's mouse and keyboard interaction to the ImGUI widgets when mouse is on top of an ImGUI window.

Firstly, I suggest familiarizing yourself with the sample programs in https://github.com/ocornut/imgui/tree/master/examples, if you've not already done that. There are so many for every single device and platform! Do not be overwhelmend, you can start with the example_glfw_opengl3 to see how ImGUI draw lists are rendered and notice the interaction code.

After you're comfortable with DearImGUI, poke inside vtkDearImGUIInjector. This will require some understanding of the VTK rendering and interaction system. In brief:

Start ImGUI draw list after vtkRenderWindow emits vtkCommand::StartEvent
Render ImGUI draw list after vtkRenderWindow emits vtkCommand::RenderEvent
Take control of the VTK interactor event loop and run indefinitely. This is done on the emission of vtkCommand::StartEvent from the interactor.
Tear down dear imgui when interactor emits vtkCommand::ExitEvent

The code is well commented where needed, please do not hesitate to ask any specific questions.

@jaswant.panchumarti : I see that imGUI is more like an application framework in vtkDearImGUIInjector, could it be integrated as an element like an actor ?

Interesting idea. so each imgui widget is rendered as a vtkActor? This is cool because you can rotate panel widgets in 3D just like vtkActors or you meant vtkActor2D?

I had 2D actors in mind, but anything is possible.

First step is getting the UI to render along with VTK's frames.

Stuck at the 1st step... Familiar with ImGui but not so much with OpenGL and roots of VTK (I use VTK from "outside" as a "user" more than a "developer" for years now): I guess this is why I feel stuck here!

I was thinking at posting "problems" on the way but there are many of them and I'am not sure how to isolate them (to make them "postable")... I'll have a fresh eye on this following your comments! :)

could it be integrated as an element like an actor ?

What do you mean could you elaborate?

Actually I am stuck at 1st step (getting the UI to render along with VTK's frames): I am just trying for now to extract GFLWwindow from vtk* classes (to use later ImGui but without using ImGui for now), but, get an X error on Render() here https://discourse.vtk.org/t/how-to-integrate-vtk-with-opengl/11526/10?u=fghoussen. I create a GFLWwindow on the one hand, and, a vtkGenericOpenGLRenderWindow and a pipeline based on it on the other hand... But didn't get how to "set" the GFLWwindow "into" vtkGenericOpenGLRenderWindow, or, how to "extract" GFLW window from vtkGenericOpenGLRenderWindow... So I guess that why I get an X error

@jaswant.panchumarti: is there a way to get a pointer to the GLFWwindow out of vtkExternalOpenGLRenderWindow?

Did you try SetWindowId and GetGenericWindowId?

Did you try SetWindowId and GetGenericWindowId?

Sorry for the late reply! I had to deal with other stuffs in between...

At this step, when I do not render I have a black screen, when I render I have an X error X Error of failed request: BadWindow (invalid Window parameter)

-      //rdrWin->Render();
+      rdrWin->Render();

I've just tried to SetWindowId

   vtkSmartPointer<vtkExternalOpenGLRenderWindow> rdrWin = vtkSmartPointer<vtkExternalOpenGLRenderWindow>::New(); // Window to render into.
+  rdrWin->SetWindowId((void *) window);

Where window is a GFLWwindow I created myself

GLFWwindow * window = glfwCreateWindow(width, height, "win", NULL, NULL);
glfwMakeContextCurrent(window);

And still get the same behavior: when I do not render I have a black screen, when I render I have the same X error

... And cherry on the cake, just rebuild VTK in debug, but, gdb doesn't even trap the X error ?!...

>> gdb --args ./vtkTest
(gdb) r
[New Thread 0x7fffdcff96c0 (LWP 15456)]
[New Thread 0x7fffbbfff6c0 (LWP 15457)]
[New Thread 0x7fffbb7fe6c0 (LWP 15458)]
[New Thread 0x7fffbaffd6c0 (LWP 15459)]
Update thread: exiting...
[Thread 0x7fffbaffd6c0 (LWP 15459) exited]
X Error of failed request:  BadWindow (invalid Window parameter)
  Major opcode of failed request:  3 (X_GetWindowAttributes)
  Resource id in failed request:  0x0
  Serial number of failed request:  8
  Current serial number in output stream:  9
[Thread 0x7fffbb7fe6c0 (LWP 15458) exited]
[Thread 0x7fffbbfff6c0 (LWP 15457) exited]
[Thread 0x7fffdcff96c0 (LWP 15456) exited]
[Thread 0x7fffdd7fa6c0 (LWP 15455) exited]
[Thread 0x7fffddffb6c0 (LWP 15454) exited]
[Thread 0x7fffde7fc6c0 (LWP 15453) exited]
[Thread 0x7fffdeffd6c0 (LWP 15452) exited]
[Thread 0x7fffdf7fe6c0 (LWP 15451) exited]
[Thread 0x7fffdffff6c0 (LWP 15450) exited]
[Thread 0x7fffe4ffe6c0 (LWP 15449) exited]
[Thread 0x7fffe57ff6c0 (LWP 15448) exited]
[Inferior 1 (process 15444) exited with code 01]
(gdb) r

Not even a way to get a clue about this !?...

Not even a way to get a clue about this !?...

@jaswant.panchumarti: I guess the SetWindowId doesn't work:

>> xtrace -n -D :10 ./vtkTest
...
001:<:0007:  4: Request(43): GetInputFocus 
001:>:0007:32: Reply to GetInputFocus: revert-to=PointerRoot(0x01) focus=0x0300000b
000:>:00e2: Event (generated) ClientMessage(33) format=0x20 window=0x0300000b type=0x170("WM_PROTOCOLS") data=0x7d,0x01,0x00,0x00,0xe8,0xba,0x7d,0x00,0xe8,0xba,0x7d,0x00,0x70,0x21,0x11,0x89,0x88,0x13,0x00,0x00;
001:<:0008:  8: Request(3): GetWindowAttributes window=0x00000000
001:<:0009:  8: Request(14): GetGeometry drawable=0x00000000
001:>:0008:Error 3=Window: major=3, minor=0, bad=0x00000000, seq=0008
001:>:0009:Error 9=Drawable: major=14, minor=0, bad=0x00000000, seq=0009
X Error of failed request:  BadWindow (invalid Window parameter)
  Major opcode of failed request:  3 (X_GetWindowAttributes)
  Resource id in failed request:  0x0
  Serial number of failed request:  8
  Current serial number in output stream:  9

AFAIU, the problem is about how to get a pointer to the GLFWwindow owned by vtkExternalOpenGLRenderWindow, or, how to set this pointer (with the one I created)

Did you try SetWindowId and GetGenericWindowId?

GetGenericWindowId is null ?!...

Code change:

   vtkSmartPointer<vtkExternalOpenGLRenderWindow> rdrWin = vtkSmartPointer<vtkExternalOpenGLRenderWindow>::New(); // Window to render into.
+  std::cout << "DEBUG - rdrWin::GetGenericWindowId " << rdrWin->GetGenericWindowId() << std::endl;

And trace at run time:

>> ./vtkTest
DEBUG - rdrWin::GetGenericWindowId 0

... And rdrWin->InitializeFromCurrentContext() doesn't help neither...

Why use vtkExternalOpenGLRenderWindow with GLFWWindow? Can you clarify your approach on getting dear imgui in VTK? VTK doesn't have GLFW integration at all.

I am used to VTK for many years now (as a C++ user - not so must as a developer, not familiar with VTK guts and internal mechanisms), familiar enough with ImGui (played with examples and more), but very new to OpenGL-related things (currently learning the hard way from https://learnopengl.com... When I started this, I was like "sounds like OpenGL means using GLFW anf GLAD"... Now only, I understand that likely GLFW is for window management only, GLAD or GLEW for mouse/keyboard management only, and, OpenGL is yet another bunch of stuffs for rendering only - unless I still missed some information along the way!...).

Learning step by step: I have to say that OpenGL is a very wide and confusing topic (at least to me) moreover when you must learn it from scratch on your own as I have to...

the problem is about how to get a pointer to the GLFWwindow owned by vtkExternalOpenGLRenderWindow, or, how to set this pointer (with the one I created)

I have just succeed (hard time!) to replace GLAD with GLEW: same problem, X error and crash (I kept using GLAD as this was the only thing I was able to deal with from my learning examples! Now I understand GLAD and GLEW do the "same" things, so I tried to replace GLAD by GLEW as GLEW seems to be used in VTK source tree).

Can you clarify your approach on getting dear imgui in VTK?

Basically, from the beginning, I was like: "OK, I run OK the ImGui examples, now let just get out of VTK a backend to pass to ImGui, let's look at ImGui examples, OK the GLFW/GLAD examples seems the most simple to me on unix/debian, OK so let's try to get out of one vtk class some kind of GLFWwindow to pass to ImGui, OK let's look in the VTK source code, OK I found a vtkExternalOpenGLRenderWindow"... At this step, I was still thinking that "OpenGL = GLFW + GLAD-or-GLEW" so this looked to me to be the good class candidate: that's why I was looking for a way to "get a pointer to the GLFWwindow owned by vtkExternalOpenGLRenderWindow". I understand now I was wrong!

My understanding at the time sounds more like "3D = GLFW + GLAD-or-GLEW + OpenGL" (not sure I get this totally right but would say you need all of them not just GLFW + GLAD-or-GLEW).

Anyway looking backwards, what I've done is:

creating a window with GLFW
managing keyboard/mouse with GLAD or GLEW
creating a VTK pipeline, and, put aside the vtkExternalOpenGLRenderWindow for later use
then following https://github.com/trlsmax/imgui-vtk, I created an event loop (as ImGui examples have) into which I was hoping both to add ImGui calls and to call "Render()" on the vtkExternalOpenGLRenderWindow I created just before: in this event loop, when calling Render(), I get an X error.

At first, when I was thinking "OpenGL = GLFW + GLAD-or-GLEW", I believe the problem may come from missing connection / get-set between "me" and "vtk". Again, that's why I was looking for a way to "get a pointer to the GLFWwindow owned by vtkExternalOpenGLRenderWindow" (to "make" this connection)

Now, as my understanding evolved more into "actually 3D = GLFW + GLAD-or-GLEW + OpenGL", I realize that this actually should work (GLFW and GLAD-or-GLEW are created by "me", the OpenGL rendering is managed by "vtk" through the vtk pipeline)... But it doesn't work: Render() call triggers a X error and crash. I don't understand why and don't find the magic fix in https://github.com/trlsmax/imgui-vtk or https://github.com/jspanchu/vtkDearImGUIInjector ?! :) In ImGui examples, "GLFW and GLAD-or-GLUE" are created on one side, and "OpenGL" is managed on the "other" side (before and in the event loop): sounds to me that that's what I have now too with vtk piepline, but, still it crashes?!

@jaswant.panchumarti: do you know why this crashes? Is this expected?

Note: hope I succeeded to be clear enough... Not easy as I learn all this on the way... That's also a reason why it not easy for me to isolate problems (difficult to post issues on vtk discourse): all questions / problems are mixing up. Indeed I try to find a way to isolate them before to be able to fix them...

Don't be tough on yourself, take it easy! No one became an expert over a week. It will take some time to get used to VTK and OpenGL as a developer.

But it doesn't work: Render() call triggers a X error and crash.

Because you're not letting VTK do the window management or setup the window. vtkExternalOpenGLRenderWindow is for really advanced use cases. Just because the DearImGUI examples use GLFW, doesn't mean you've to also use GLFW. VTK abstracts away windowing logic just like GLFW. Avoid GLFW integration as that is very complex. Use the good old vtkRenderWindow and vtkRenderWindowInteractor. These two will provide you everything that is needed to overlay content or get events. This is the 'magic'.

Were you able to build any of those two VTK-imgui projects? Take some time to run the executables from vtkDearImGUIInjector and imgui-vtk in the debugger and step through to see how it works. VTK uses X for windowing on Linux. Use a debug build of VTK and step through VTK code as well. Reading the code alone is not enough to understand how something works.

No one became an expert over a week.

;)

Were you able to build any of those two VTK-imgui projects? Use a debug build of VTK and step through VTK code

Yes and yes!... But likely get lost on the way: not easy to follow the track when you're missing a compass! :)

Use the good old vtkRenderWindow and vtkRenderWindowInteractor

Waow!... No more crash: magic! :) But get a black screen with nothing rendered in it?!... The VTK pipeline is not rendered: do you get why?

@jaswant.panchumarti : I'm curious, why the external render window + GLFW would not work ? We are doing exactly that in F3D examples (minus imGUI of course) and it works perfectly.

It will work with correct setup which is what I think they are missing.

No more crash: magic! :) But get a black screen with nothing rendered in it?!...

Initially walked away from good old vtkRenderWindow because of this:

~/imgui-vtk >> git grep vtkGenericOpenGLRenderWindow
VtkViewer.cpp:  renderWindow = vtkSmartPointer<vtkGenericOpenGLRenderWindow>::New();
VtkViewer.h:#include <vtkGenericOpenGLRenderWindow.h>
VtkViewer.h:    vtkSmartPointer<vtkGenericOpenGLRenderWindow> renderWindow;
VtkViewer.h:    inline void setRenderWindow(const vtkSmartPointer<vtkGenericOpenGLRenderWindow>& renderWindow) {
VtkViewer.h:    inline vtkSmartPointer<vtkGenericOpenGLRenderWindow>& getRenderWindow() {

@jaswant.panchumarti: Black screen in connection with misuse (or not using) of frame buffers?

~/imgui-vtk >> git grep vtkOpenGLFramebufferObject
VtkViewer.h:#include <vtkOpenGLFramebufferObject.h>


~/imgui-vtk >> git grep fbo
VtkViewer.cpp:  auto vtkfbo = renderWindow->GetDisplayFramebuffer();
VtkViewer.cpp:  vtkfbo->Bind();
VtkViewer.cpp:  vtkfbo->UnBind();

Following imgui-vtk, tried to add vtkfbo->Bind() and/or vtkfbo->UnBind(): no crash, black screen

Following imgui-vtk, tried to add

   rdrWin->InitializeFromCurrentContext();
+  rdrWin->SwapBuffersOn();
+  rdrWin->SetOffScreenRendering(true);
+  rdrWin->SetFrameBlitModeToNoBlit();

no crash, black screen.

Something seems to be missing.

No more crash, but, black screen?! Sounds like rendering is not done?!... @jaswant.panchumarti: any clue?!... Dead-end?...

Printing "Render" before each rdrWin->Render(); gives:

>> xtrace -n -D :10 ./vtkTest
Render
000:<:00e0: 72: Present-Request(148,1): Pixmap window=0x01200009 pixmap=0x01200011 serial=5 valid=0x00000000 update=0x00000000 x_off=0 y_off=0 target_crtc=0x00000000 wait_fence=0x00000000 idle_fence=0x01200012 options=0 target_msc=6019242046521344 divisor=0 remainder=0 notifies=;
000:>:00e0: Event Generic(35) Present(148) IdleNotify(2) event=0x0120000c window=0x01200009 serial=5 pixmap=0x01200011 idle_fence=0x01200012
000:>:00e0: Event Generic(35) Present(148) CompleteNotify(1) kind=Pixmap(0x00) mode=Copy(0x00) event=0x0120000c window=0x01200009 serial=5 ust=8114833749327216645 msc=6019499744559104

No more crash, but, black screen?! Sounds like rendering is not done?!...

Following this with gdb, seems no glDraw* call is triggered: is this expected? Up to here, from a regular classic vtk pipeline (that worked), I've tried to set GLFW+GLEW "myself" and let vtk do the OpenGL (gl*) stuffs for me: so that I expected vtk to handle the glDraw* calls. Is this expectation wrong?

Any clue is appreciated! ;)

@jaswant.panchumarti: could you explain/detail what does and why these AddObserver calls? What vtkNew<vtkCallbackCommand> EventCallbackCommand is meant for and how to use it?

Are there doc or minimal examples to understand how to use them and why/when these may be needed? I feel like these may explain black screen I observe : are all these callbacks supposed to trigger glDraw calls?

I'd start by building @jaswant.panchumarti prototype and make it work with VTK master, then see how it could be integrated easily, probably through a dedicated actor/renderer/mapper ?

ImGui integration?

Designs

Child items 0

Activity