Magic leap and Mixed Reality Lightfield : augmented reality is coming to rock you

Magic Leap is a US startup company that is working on a head-mounted virtual retinal display which superimposes 3D computer-generated imagery over real world objects, by projecting a digital light field into the user's eye. It is attempting to construct a light-field chip using silicon photonics.

First of all, "Mixed Reality" is a marketing term, where as "Lightfield" is a scientific term. In this case, Mixed Reality basically means augmented reality that is seamlessly integrated into the environment. Basically, you see "holograms" (Microsoft's marketing term of choice) that ideally look as close as possible to real objects. When Magic Leap says that their technology produces Mixed Reality Lightfield, they means they utilize lightfields for compelling mixed reality.

So what is a lightfield? In the most literal sense, a lightfield is a representation of all the light rays from an object or a scene that reaches a place in space. In contrast to a photograph or screen, which only shows a fixed set of light rays from a single perspective, a lightfield has the entire scene recreated. For example, if you watched a movie on a light field monitor, it would be like seeing the scene through a window. You could focus on different objects in the scene and you  would see a slightly different perspective as you moved your head around.

Now in reality, producing a light field is ridiculously over computationally complex and produces significantly more data than is ever necessary.  For a head mounted display, even less of this data is necessary, as your head is in a pretty fixed position vs. the display.

So here's where Magic Leap comes in. Their headset uses a scanning laser projector to project a digital simplification of a light field "directly" on your retina. The projector works by rapidly vibrating a laser to displaying each individual pixel. In addition to adjusting the X,Y position of each pixel, these vibrations can also adjust the Z position of each pixel, aka where you focus on it. This allows virtual objects to look like they're located where they're actually supposed to be located, which solves a whole host of problems that traditional stereoscopic head mounted displays suffer from, such as nausea and eye strain.