I recently received a pair of AirPod Pros as a gift. I was happy with my regular AirPods but pleasantly surprised by the noise cancellation options offered by the new Pro version. There is a little bit of Apple magic happening when you're in a louder environment and you toggle your headphones between the various modes. When off, the AirPods block a lot of the sound around you just by fitting snuggly in your ears. But, when you turn on noise cancellation, the world around you fades out and you can focus on any audio you're playing. Transparency mode is even cooler. The headphones listen to the sounds in the environment and then play that sound over the speakers in a way that sounds as if you aren't wearing headphones at all. You can wear the AirPods Pro and hold a conversation as if you didn't have something in your ears.
This Transparency mode is going to be the key to Apple's first VR headset. Instead of mimicking the sounds around you, the VR headset will mimic the visual world around you. The AirPods Pro are convincing enough in their reproduction that your brain feels like you're not wearing headphones. The VR headset will convince your brain that you are wearing "glasses" or a transparent visor rather than a solid screen.
Just like the AirPods Pro have microphones to listen, the VR headset will have cameras to capture the world. Apple will fine tune the latency and position tracking to the point where your brain just won't be able to tell if you are seeing the world with or without the headset.
The opaque VR headset solves a big problem of how to overlay AR objects on the real world. If the lenses were transparent, they need to allow light through to the eyes while simultaneously blocking light for AR objects and matching that lighting to the real world. If you've used a Microsoft Hololens, you know how difficult it can be to create this effect for objects that are both bright and dark at the same time.
When a user puts on the headset and AirPods Pro, they will get a pass-through version of the world. The lidar systems on current Phones and iPads offer a look into Apple's efforts to inject AR objects into camera views. You won't worry about bumping into furniture or doors while wearing the headset because they will be visible to you. Movement in transparency mode will feel natural.
To observers, this will look odd. The wearer of the headset will act is if they aren't wearing one at all but the observer won't see their eyes and facial expressions. Apple could offer an external screen on the front of the headset that works in the opposite direction of the internal pass-through. Cameras on the inside of the headset could project the wearer's face and eyes on a screen with a high enough resolution to be convincing to observers.
The headset will also have the visual equivalent of noice cancellation. When the wearer wants to be totally immersed in a virtual environment, they can toggle the settings to fade out the camera pass-through and just operate like other VR headsets.
The first version of the headset won't be perfect but one or two aspects will be so compelling that it will work. Just like the first iPhone didn't have an app store, the first headset will have limitations. It may require a power cord or have fairly short battery life at first. The field of view may be limited. Over time, the technology will continue to improve and developers will create new interesting use cases with innovative software.
I wasn't excited about the idea of an opaque VR headset from Apple until I tried the transparency mode on AirPods Pro. Now I get where Apple is going and I think they're going to have a winner that will change the public perception of VR/AR from a gimic into something practical for humans.
Read More