I [James Darpinian] recently purchased two commercial eye trackers: the Tobii EyeX and the Eye Tribe tracker, both $99. I’m excited about the possibilities for eye tracking as an input method and for telepresence, but the reality of these trackers is disappointing. The accuracy is so bad that they can’t possibly be used as mouse replacements. To make a click target large enough to hit reliably, it would have to be about 1/4 of the screen width and height; anything else would just be too small.
These trackers are trying to solve a harder problem than they need to, though. They’re several feet away from the eyes, and they have to deal with arbitrary head motion and variable lighting. For virtual reality, we can fix all of these problems by building eye tracking into a head-mounted display. The camera can be fixed to the head and mere inches from the user’s eye, in a completely controlled lighting environment. Accuracy can be much better!
Having eye tracking in a head-mounted display is attractive for several other reasons. It helps solve the VR input problem by adding a built-in input method roughly equivalent to a mouse pointer. There are a lot of possibilities for interesting game mechanics driven by eye tracking. It also helps solve the face-covering problem: when you’re wearing a head-mounted display, your facial expressions are hidden. Eye tracking records the hidden part of your face, and combined with external cameras can recover your entire facial expression so that you can have natural conversations with other people in VR.
So, after the disappointing performance of the commercial eye trackers, I decided to try making a VR eye tracker of my own. Here’s how I did it: