Meta Has A Secret VR Headset That May Have Significant Advantages Over Apple’s Vision Pro

All mixed-reality (and some virtual-reality) headsets have one thing in common, but none are perfect yet. It’s a passthrough. By precisely arranging the camera and lens, metaquest pro, apple vision pro,Furthermore $299 Quest 2 They can simulate the physical environment to varying degrees as they obscure the real world view.

Also: Apple’s Vision Pro Labs Can Hear Developers Gasping, The Company Says

Passthrough enables heavy computational processing for augmented reality applications such as home design, fashion, and interactive learning, use cases with some degree of real-world spatial awareness, and image simulation.

Naturally, how things look in real life, especially considering the distance between where the headset’s camera is set and where your “camera” (or eyes) are located. Achieving a visual experience similar to this is not an easy task. . But Meta has a secret weapon within her Reality Labs Display Systems Research (DSR) team that the pass-through he believes cracked the code.

Flamera is a pre-production mixed reality headset that uses “light field passthrough” as a means of replicating an environment exactly. By assembling a grid of apertures between the camera’s lens array and the sensor, optimal light capture is achieved no matter how the eye moves, naturally recording both the distance and movement of surrounding objects. .

This aperture module gives you what can only be described as a bug’s eye look with holes in all directions and angles. According to Meta, the headset takes the raw sensor data captured by all these holes and stitches it together through depth-aware reconstruction. Below is a comparison of passthrough mode and reality.

“The Flamera’s optical design works best when the headset is thin, which allows the pass-through camera to be as close as possible to the user’s eyes,” said a research scientist on Meta’s DSR team. Grace Kuo said. DSR Director Douglas Lanman added, “Our lightfield passthrough research is focused on previewing his MR passthrough experience with perspective correction.”

Just last month, Flamera was demonstrated at Siggraph 2023, an annual convention showcasing innovators in computer graphics and interactive technology. “It was definitely of the highest quality. [augmented reality] It was a pass-thru like we’ve never seen before,” attendee Anshel Sag, principal analyst at Moors Insights & Strategy, told IEEE Spectrum.

Also: Meta Quest 3 announced: price, features and when you can buy it

Sag also noted that the beta headset has a narrow field of view, which could be a problem for VR/AR applications that need to look left and right.

Still, Flamera could be the start of something big. And with a flurry of new mixed-reality headsets set to be released in the coming months, culminating in the launch of Apple’s Vision Pro early next year, the technology his DSR team at Meta has developed is worth watching. It has become a kimono.