World Coordinate Picking

Manipulating an object's world coordinates with OpenGL is tricky. This is an operation which involves translating the screen coordinates to a position in the world. The "tricky" part of this is determining the depth of the object in the range [0,1] to use as the wz argument to gluUnProject (a function that does the hard screen coordinate to world coordinate transformation for us).

Unfortunately, googling for gluUnProject yields some very bad results. For example the NeHe tutorial gives a wretched example using glReadPixels with GL_DEPTH_COMPONENT as the format. This is a really, really stupid idea. First of all, imagine an object is in focus (I've selected it as the target from a drop down menu, or some other way), and then I click on some arbitrary point on the screen. Well, guess what kids? I get the window depth at that point (not the point corresponding to the object I'm interested in). Even if I point directly over the object, it's easy for the mouse to race ahead of the object while dragging it across the screen, and (boom!) the window z depth is 1.0, or the far clipping plane, and my object flies of the friggin' screen. Beautiful!

Other links that crop up literally hard code the wz depth value which only makes sense if you're manipulating an object sitting directly on the near clipping plane.

So, here's how you do it, and it's very simple. To determine the window depth of the object, first do the reverse of gluUnProject (that is, gluProject) based on the coordinates of the object in focus. Here's my final (working) code in python:

        viewport = (GLint * 4)()
        mvmatrix = (GLdouble * 16)()
        projmatrix = (GLdouble * 16)()
        glGetIntegerv(GL_VIEWPORT, viewport)
        glGetDoublev(GL_MODELVIEW_MATRIX, mvmatrix)
        glGetDoublev(GL_PROJECTION_MATRIX, projmatrix)
        wx = GLdouble()
        wy = GLdouble()
        wz = GLdouble()
        sz = GLdouble()
        ex = target.pos.x
        ey = target.pos.y
        ez = target.pos.z
        gluProject(ex, ey, ez, mvmatrix, projmatrix, viewport, wx, wy, sz)
        gluUnProject(x, y, sz,
          mvmatrix, projmatrix, viewport, wx, wy, wz)

        ...
        target.pos.x = wx.value
        target.pos.y = wy.value