VR

Back in October 2022, I predicted that Apple would be using a transparency mode similar to the audio transparency mode in AirPods Pro. Today, Apple showed off the Apple Vision Pro to the public. My prediction turned out to be just about right on the money. As a user, you can choose to switch the headset between a virtually transparent view of the world around you or switch it to be opaque when you want a more VR experience. What I didn't expect was the ability to dial up/down the amount of the world that is visible. Not only is that super helpful to feel connected, it also lets software adjust the transparency as needed. Apple showed off the ability to detect people nearby and let the people fade into the closed off VR world of the headset so you aren't surprised while using the device. They also did put a screen on the outside to show a representation of your eyes so that conversing while wearing the device is more natural for the people not wearing one.

The external battery pack, the size, and other limitations show that this is a 1.0 device. I remember getting an original iPhone and paying what seemed like a crazy price at the time. You couldn't even download apps on the first iPhone! It also didn't have a front camera, no copy/paste, and was tied to a single carrier, Cingular, with less than a 3G network. Apple Vision Pro is in that phase.

It will get lighter. It will get smaller. It will gain new abilities. It will get wireless support. There will be more style revisions every year. There will an accessory market to personalize the device to your personality. The eco system will grow and others will copy Apple with varying degrees of success.

I'm most excited to see what developers do with the device once they've had a chance to understand all of the hardware. I'm sure we'll see an app that allows realtors to just walk through a house and instantly create a 3D model of the space to share with clients. There will be someone that creates the equivalent of a flashlight app and probably make a lot of money for a very simple program that uses the hardware in a novel way. It's an exciting development and I'm sure this is only the start.

I recently received a pair of AirPod Pros as a gift. I was happy with my regular AirPods but pleasantly surprised by the noise cancellation options offered by the new Pro version. There is a little bit of Apple magic happening when you're in a louder environment and you toggle your headphones between the various modes. When off, the AirPods block a lot of the sound around you just by fitting snuggly in your ears. But, when you turn on noise cancellation, the world around you fades out and you can focus on any audio you're playing. Transparency mode is even cooler. The headphones listen to the sounds in the environment and then play that sound over the speakers in a way that sounds as if you aren't wearing headphones at all. You can wear the AirPods Pro and hold a conversation as if you didn't have something in your ears.

This Transparency mode is going to be the key to Apple's first VR headset. Instead of mimicking the sounds around you, the VR headset will mimic the visual world around you. The AirPods Pro are convincing enough in their reproduction that your brain feels like you're not wearing headphones. The VR headset will convince your brain that you are wearing "glasses" or a transparent visor rather than a solid screen.

Just like the AirPods Pro have microphones to listen, the VR headset will have cameras to capture the world. Apple will fine tune the latency and position tracking to the point where your brain just won't be able to tell if you are seeing the world with or without the headset.

The opaque VR headset solves a big problem of how to overlay AR objects on the real world. If the lenses were transparent, they need to allow light through to the eyes while simultaneously blocking light for AR objects and matching that lighting to the real world. If you've used a Microsoft Hololens, you know how difficult it can be to create this effect for objects that are both bright and dark at the same time.

When a user puts on the headset and AirPods Pro, they will get a pass-through version of the world. The lidar systems on current Phones and iPads offer a look into Apple's efforts to inject AR objects into camera views. You won't worry about bumping into furniture or doors while wearing the headset because they will be visible to you. Movement in transparency mode will feel natural.

To observers, this will look odd. The wearer of the headset will act is if they aren't wearing one at all but the observer won't see their eyes and facial expressions. Apple could offer an external screen on the front of the headset that works in the opposite direction of the internal pass-through. Cameras on the inside of the headset could project the wearer's face and eyes on a screen with a high enough resolution to be convincing to observers.

The headset will also have the visual equivalent of noice cancellation. When the wearer wants to be totally immersed in a virtual environment, they can toggle the settings to fade out the camera pass-through and just operate like other VR headsets.

The first version of the headset won't be perfect but one or two aspects will be so compelling that it will work. Just like the first iPhone didn't have an app store, the first headset will have limitations. It may require a power cord or have fairly short battery life at first. The field of view may be limited. Over time, the technology will continue to improve and developers will create new interesting use cases with innovative software.

I wasn't excited about the idea of an opaque VR headset from Apple until I tried the transparency mode on AirPods Pro. Now I get where Apple is going and I think they're going to have a winner that will change the public perception of VR/AR from a gimic into something practical for humans.