Xlib + openGL programming and multithreading

When programming using Xlib or Win32 the usual program is something like that:

setupWindow()
setupOpenGL()

while(exit)
{
  while(haveEvent()){
    event = GetEvent()
    processMessage(event)
  }
  applicationLogic()
  renderScene()
}

Where in the processMessage() will manage the event like key pressed or mouse moved and applicationLogic() update the application state (like physics simulation). This pattern have two disadvantages

• both the processMessage() and applicationLogic() function can be slow and freeze the rendering (for example when a press return I want to load a picture).
• every frame check in polling if there are some messages, nobody like polling.

Developing a framework the problem is quite common cause you don’t know what the user will do with your framework, the only thing you know is that you don’t want to freeze the rendering.
To make the long story short, we have to move the renderScene() function in another thread. It look incredible but I didn’t found any tutorial about this, so here it is.
I want to create an application that create n openGl windows, each window will have it’s own thread.
Something like this:

for(i = 0; i < numWindow; ++i){
  win = setupWindow()
  createRenderThread()
}
while(ended)
{
   event = waitForEvent() // this thread sleep if there aren't event
   processMessage(event)
}

I tried exactly what I wrote above but I got a lot of multi thread issue. If you use multiple context openGl it’s ok with multithreading, is you use the magic function XInitThread also xlib is thread safe, the problem in GLX. It look like that have a bug that freeze the swapbuffer calls when another thread is waiting for events, when they both try to get the display they freeze. I tried different solution (using xcb works on some platform), but the most reliable is to use multiple X connection.
Ok, enought talk, let’s start with the code. I suppose that you already know some basic Xlib programming, so I wont explain basic function call.
First of all we need a structure to contain the shared memory between the main thread and the render thread.

struct RenderData {
  Display d_;   // Display connection used for this thread
  Atom delMgs_; // Explain later
  GLXDrawable surface_;
  GLXContext  ctx_;
  // Other data
  Color bg_;     // background color
  Size winSize_; // Windows dimension
};

then we need to initialize the main x connection.

int main(){
// We must call this to make X "thread safe"
  XInitThread();
// opening the connection with X
  Display *display = XOpenDisplay();
  if(display == NULL) return -1; // Add meaning full message error here
  int xScreen = XDefaultScreen(display);
// Let's choose a visual (pretty standard function, link to the full code below)
  GLXFBConfig *fbConfig = chooseFBConfig(display, screen);
  XVisualInfo* visInfo = glXGetVisualInfoFromFBConfig(display, fbConfig[0]);
  Window root = XRootWindow(display, screen);
  ....

Now we must create an Atom to get informed about window destruction (when the user press the X button).

Atom delMsg = XInternAtom(display, "WM_DELETE_MESSAGE", True);
...

Now we can create the windows:

RenderData param[numWindow];
for(int i = 0; i < numWindow; ++i){
// create a X connection for each window
   param[i].d_ = XOpenDisplay(NULL);
   param[i].delMsg_ = delMsg;
   XSetWindowAttributes winAttrib;
// Need to create a color map for each connection
   winAttrib.colormap = XCreateColormap(param[i].d_, root, visInfo->Visual, AllocNone);
// Create the window and the openGl context in the standard way
   param[i].surface_ = XCreateWindow(param[i].d_, root, 0, 0, 300, 200, 0, visInfo->depth, InputOutput, visInfo->visual, CWColormap, &winAttrib);
   param[i].ctx_ = glXCreateContext(param[i].d_, visInfo, NULL, True);
// Now the trick, tell X that we want to retrieve event for the window in the main connection
   XSelectInput(display, param[i].surface_, StructureNotifyMask);
// Register the Atom for this window
   XSetWMProtocolos(param[i].d_, param[i].surface_, &delMsg, 1);
// Create a new thread (c++0x way)
   param[i].thread_ = new thread(render, param+i);
// Keep track of the association window <-> data
   wMap.insert(std::make_pair(param[i].surface_, r));
}

The thread function will create a thread that will execute the function render using param+i as parameter.
The render function will simply cycle endlessly until someone press the X button.

void render(RenderData *data){
   XEvent event;
// show the window
   XMapWindow(data->d_, data->surface_);
// Each thread must have it's current context
   glXMakeCurrent(data->d_, data->surface_, data->ctx_);
   glClearColor(data->bg.red_, data->bg.green_, data->bg.blue_, 1.f);
   while(1){
      if(XPending(data->d_){
         XNextEvent(data->d_, &event);
// Got death message
         if(event.type == ClientMessage && event.xclient.data.l[0] == data->delMsg)
             break;
      }
      glViewport(0, 0, data->winSize_.w, data->winSize_.h_);
      // do some rendering here
// Show our beautiful rendering
      glXSwapBuffers(data->d_, data->ctx_);
      usleep(16000);
   }
// Work finished, destroy the context, the window
   glXDestroyContext(data->d_, data->ctx_);
   XDestroyWindow(data->d_, data->surface_);
// and close the connection
   XCloseDisplay(data->d_);
}

The render function render the scene and pool for the close even. In this way we are sure that our render will not freeze.
We need the last part of the code, the event handling. After the for that create the windows…

int windowLeft = numWindow;
while(windowLeft > 0){
  RenderData *data;
  XEvent event;
  XNextEvent(display, &event); // blocking function
  switch(event.type){
  case ConfigureNotify: // window changed size or position
    // get the data associate to the window
     data = selectData(wMap, event.xconfigurerequest.window);
     data->winSize.w = event.xconfigurerequest.width;
     data->winSize.h = event.xconfigurerequest.height;
     break;
  case DestroyNotify: // the window has been destroyed
     data = selectDatta(wMap, event.xdestroywindow.window);
// wait the thread to end
     data->thread_.join();
// delete the thread
     delete(data->thread);
// remove the thread from the data
     wmap.erase(event.xdestroywindow.window);
     windowLeft--;
     break;
  }
}

The StructureNotify we have request to the XSelectInput give us these message: ConfigureNotify, when the window change size or position; DestroyNotify, when the window is destroyed; and other message like Map/Unmap that we are not using.
In this code there is a possible data corruption issue caused by writing in a non atomic way the window size.

Here what you are waiting for, the whole source of the example.

#include <X11/X.h>
#include <X11/Xlib.h>
#include <GL/glx.h>
#include <GL/gl.h>
#include <thread>
#include <map>

using namespace std;
GLXFBConfig *chooseFbConfig(Display *d, int screen);

static const int totalThread = 3;

struct Size{
   Size() {}
   Size(unsigned int x, unsigned int y):w(x), h(y){}
   unsigned int w, h;
};

struct Color{
   Color(){}
   Color(unsigned int c):
       red((0xFF&c)/255.f),
       green((0xFF&(c>>8))/255.f),
       blue((0xFF&(c>>16))/255.f)
   { }
   Color(float r, float g, float b): red(r), green(g), blue(b) {}
   float red, green, blue;
};

struct RenderData{
   RenderData() : exit(false), thread_(NULL) {}
   Display *d_;
   Atom delMsg;
   GLXContext ctx;
   GLXDrawable win;
   Color bg;
   Size winSize;
   bool exit;
   thread *thread_;
};

typedef std::map<Window, RenderData*> WinDataMap;

RenderData* selectData(WinDataMap& map, const Window& w){
   WinDataMap::iterator it = map.find(w);
   if(it == map.end())
      throw runtime_error("invalid window");
   return (it->second);
}

void render(const RenderData *data)
{
   XMapWindow(data->d_, data->win);
   glXMakeCurrent(data->d_, data->win, data->ctx);
   glClearColor(data->bg.red, data->bg.green, data->bg.blue, 1.0);
   while(!data->exit)
   {
      XEvent event;
      if(XPending(data->d_)){
         XNextEvent(data->d_, &event);
         if(event.type == ClientMessage && static_cast<unsigned int>(event.xclient.data.l[0]) == data->delMsg){
            puts("got dead event.. ._.");
            break;
         }
      }
      glViewport(0, 0, data->winSize.w, data->winSize.h);
      glClear(GL_COLOR_BUFFER_BIT);
      glBegin(GL_QUADS);
         glVertex2d(0.0f, 0.0f);
         glVertex2d(1.0f, 0.0f);
         glVertex2d(1.0f, 1.0f);
         glVertex2d(0.0f, 1.0f);
      glEnd();
      if(glGetError() != GL_NO_ERROR)
         puts("gl error");
      glXSwapBuffers(data->d_, data->win);
      usleep(16000);
   }
   glXDestroyContext(data->d_, data->ctx);
   XDestroyWindow(data->d_, data->win);
   XCloseDisplay(data->d_);
}
const unsigned int colors[totalThread] = {
   0x62E200,
   0xFF6F00,
   0x7C07A9
};

int main(int argc, char *argv[])
{
   Display *d;
   XInitThreads();
   WinDataMap wMap;

   RenderData param[totalThread];   // window parameter
   d = XOpenDisplay(NULL);
   if(d == NULL){
      puts("Unable to connect to X");
      return 1;
   }
   int screen = XDefaultScreen(d);

   Atom delMsg = XInternAtom(d, "WM_DELETE_WINDOW", True);
   XSetWindowAttributes windowAttr;
   GLXFBConfig *fbConfigs = chooseFbConfig(d, screen);
   XVisualInfo *visInfo = glXGetVisualFromFBConfig(d, fbConfigs[0]);
   Window root = XRootWindow(d, screen);
   for(int i = 0; i < totalThread; ++i)
   {
      param[i].d_ = XOpenDisplay(NULL);
      param[i].delMsg = delMsg;
      param[i].bg = Color(colors[i]);
      param[i].winSize = Size(300, 200);

      windowAttr.colormap = XCreateColormap(param[i].d_, root, visInfo->visual, AllocNone);
      param[i].win = XCreateWindow(param[i].d_, root, 200,200, 
                   300,200, 0, visInfo->depth, InputOutput, visInfo->visual,
                   CWColormap,
                   &windowAttr);
      param[i].ctx = glXCreateContext(param[i].d_, visInfo,  NULL, True);
      XSelectInput(d, param[i].win, StructureNotifyMask);
      XSetWMProtocols(d, param[i].win, &(delMsg), 1);
      printf("win %lu\n", param[i].win);
      wMap.insert(std::make_pair(param[i].win, param+i));
      param[i].thread_ = new thread(render, param+i);
   }
   XFree(visInfo);
	XFree(fbConfigs);
   int windowLeft = totalThread;
   while(windowLeft > 0)
   {
      RenderData *pData;
      XEvent event;
      XNextEvent(d, &event);
      switch(event.type){
   // Structure notify
      case ConfigureNotify:   // size or position change
         pData = selectData(wMap, event.xconfigurerequest.window);
         printf("configure win %lu: pos [%d, %d], size [%d, %d]\n", event.xconfigurerequest.window,
                event.xconfigurerequest.x,
                event.xconfigurerequest.y,
                event.xconfigurerequest.width,
                event.xconfigurerequest.height
               );
         // Warning this is not thread safe
         pData->winSize.w = event.xconfigurerequest.width;
         pData->winSize.h = event.xconfigurerequest.height;
         break;
      case CirculateNotify:
         puts("CirculateNotify"); break;
      case DestroyNotify:
         puts("DestroyNotify");
         pData = selectData(wMap, event.xdestroywindow.window);
         pData->thread_->join();
         delete (pData->thread_);
         wMap.erase(event.xdestroywindow.window);
         windowLeft--;
         break;
      case GravityNotify:
         puts("GravityNotify"); break;
      case ReparentNotify:
         puts("ReparentNotify"); break;
      case MapNotify:
         printf("Map win %lu\n", event.xmap.window);
         break;
      case UnmapNotify:
         printf("UnMap win %lu\n", event.xunmap.window);
         break;
   // Unhandled message
      default:
         printf("Type %d\n", event.type);
         break;
      }
   }
   XCloseDisplay(d);
   return 0;
}

GLXFBConfig* chooseFbConfig(Display *d, int screen) {
   int glAttrib[] = {
      GLX_DRAWABLE_TYPE , GLX_WINDOW_BIT,
      GLX_RENDER_TYPE   , GLX_RGBA_BIT,
      GLX_RED_SIZE      , 1,
      GLX_GREEN_SIZE    , 1,
      GLX_BLUE_SIZE     , 1,
      GLX_ALPHA_SIZE    , 8,
      GLX_DEPTH_SIZE    , 24,
      GLX_DOUBLEBUFFER  , True,
      None
   };
   int num; 
   GLXFBConfig *fbConfigs = glXChooseFBConfig(d, screen, glAttrib, &num);
   if(fbConfigs == NULL) throw runtime_error("Unable to find a frame buffer config!");

   return fbConfigs;
}

To compile use:
gcc glThread.cpp -std=c++0x -pthread -l stdc++ -l X11 -l GL -o glthread

–enjoy