Xtrar Solutions

 Apple Vision Pro won't solve the problems with virtual reality

Apple’s Vision Pro headset announced at WWDC 2023 blends virtual and real-world experiences together. But it still has many of the same problems as other headsets

Apple’s newly announced virtual reality headset promises to blend the real world with video and audio, ranging from immersive FaceTime video chats to watching films and shows on a huge virtual movie screen. But even the company that pioneered the modern smartphone may not have great expectations for its $3499 device at a time when rival Silicon Valley giants such as Meta and Microsoft have struggled to make VR go mainstream.

“Apple’s headset is both experimental and expensive,” says Lee Vinsel, a historian of technology at Virginia Tech. “The same was true for many other eventually successful devices, including the iPhone, but those technologies were opening up new spaces, whereas Apple is entering well-trod ground where others have failed.”

Apple’s Vision Pro headset presented at its annual Worldwide Developers Conference on 5 June is designed to deliver blended reality experiences for many familiar apps made by Apple, Microsoft and other companies that mix virtual and physical spaces for work and entertainment. The headset also provides a “see-through” experience that shows the wearer’s eyes and allows for interaction with other people in the physical world even while the wearer is interacting with virtual experiences. It even creates a digital persona to replicate the appearance of the wearer for use in FaceTime conversations and other experiences.

Scheduled to become available for purchase in early 2024, the headset is connected by a woven cable to a pocket battery that supports up to 2 hours of use and supposedly runs almost silently and at a comfortable temperature. It also contains a new Apple computer chip called R1 that processes information from 12 cameras, five sensors and six microphones in an attempt to eliminate sensor lag. The headset can be controlled solely through the wearer’s visual gaze, voice and small hand gestures such as pinching and flicking motions.

Silicon Valley has been trying to make some version of XR – the catchall phrase for virtual reality, augmented reality and mixed reality – go mainstream for decades, says Wagner James Au, author of the book Making a Metaverse That Matters. Apple’s Vision Pro headset is technically a mixed reality device that enables wearers to access immersive virtual experiences while still seeing the outside world.

Part of the challenge for XR headsets has been the steep cost, along with the physical discomfort and awkwardness of using bulky headsets. But another significant issue that has been overlooked by many tech companies involves the many people who experience virtual reality sickness, which is like motion sickness, says Au.

Despite having sold 20 million Quest headsets so far, Meta has struggled to keep VR users returning to the virtual experiences available in its previously touted metaverse platform. Shortly before Apple’s headset announcement, Meta CEO Mark Zuckerberg announced his company’s upcoming Quest 3 VR headset that also offers a mixed reality experience.

Meanwhile, Microsoft had sold just 300,000 units of its HoloLens headset as of late 2022 while similarly failing to gain traction for its own metaverse vision.

Virtual reality and other XR headsets failed to gain significant traction even during the pandemic when millions of people stayed closer to home – global shipments of both VR and AR headsets shrank by more than 12 per cent between 2021 and 2022. “The base of VR users remains small and is still almost wholly in video games and other forms of entertainment,” says Vinsel.

Given that backdrop, Apple has projected sales of less than 100,000 units of its headset and total production of potentially just 300,000 headsets, according to TrendForce. If the headset doesn’t sell well even based on those modest goals, Apple could still “present it as a prototype or as an early adopter thing” and market it primarily for business customers, says Au.

Apple’s headset development has cost the company more than $1 billion per year, according to Bloomberg. But as the world’s largest company and profits of almost $100 billion in 2022, the company can afford to bet on a multibillion dollar VR experiment even if it completely fails.

“They could throw a billion dollars at research in VR and not even notice it,” says Au. “That’s like the change of their couch.”

What it’s like wearing Apple’s buzzy Vision Pro goggles and venturing into the weird and isolating world of virtual reality

Reporters are a skeptical bunch, so it was unusual to hear so many of them raving about their firsthand experience with Apple’s next Big Thing: the high-priced headset called Vision Pro, a device infused with totally virtual reality as well as augmented reality that projects digital images on top of real-world settings.

But after wearing the Vision Pro during a half-hour demonstration meticulously orchestrated by Apple, I joined the ranks of those blown away by all the impressive technology Apple has packed into the goggles-like headset. Still, that excitement was muted by a disquieting sense of having just passed through a gateway that eventually will lead society down another avenue of digital isolation.

THE POTENTIAL UPSIDES

But first the good stuff: Vision Pro is a highly sophisticated device that is fairly easy to set up and incredibly intuitive to use. The setup requires using an iPhone to automatically take some assessments of your eyes and ears. If you wear prescription glasses (I wear contacts) some additional calibration will be needed, but Apple promises that won’t be complicated.

Once that’s all done, you will quickly find that putting on the Vision Pro is also simple, thanks to a knob on the side that makes it easy to ensure a the headset fits comfortably. And unlike other headsets, the Vision Pro isn’t an awkward-looking piece of nerdware, although the goggles aren’t exactly chic, despite looking a bit like something you might see people wearing on a ski slope, jet fighter or race car.

Controlling the Vision Pro is astoundingly easy. Users just press a button above the right goggle to pull up a virtual screen of apps, including familiar standbys for photos, messaging, phone calls, video streaming and web browsing. Opening an app just requires looking straight at it, then pinching a thumb and finger together. The same app can be closed with a finger pinch or can be moved to the side by holding two fingers together and moving them in the direction where you want to place it.

Not surprisingly, Apple’s well-curated demonstration cast the Vision Pro in the best-possible light. The headset clearly seems like it could be quite popular for business purposes, improving productivity, collaboration and video conferencing, especially in an era when more work is being done remotely.

Without causing the disorienting effects common in other virtual-reality headsets, the Vision Pro can immerse you in stunning visuals, 3-D displays of faraway places. It can insert you into videos of past memories recorded with one of the device’s 12 cameras (the demo included heartwarming scenes of a child’s birthday party and a campfire scene). It can make watching a 3-D movie, such as the latest Avatar film, feel like you are sitting in an IMAX theater while relaxing on your own couch. It can thrust you into surreal moments (at one point, I watched in wonder as a butterfly first shown in a virtual screen depicting a prehistoric era seemingly fluttered across the room and landed in my outstretched hand as I sat on a couch).

And the demo featured just enough glimpses of the way sporting events appear through the goggles to realize that the powers that be in professional and collegiate football, basketball, baseball and hockey are bound to find ways to incorporate the technology into subscription services that make viewers feel like they are sitting in the front row.

To Apple’s credit, the Vision Pro is also designed in a way that allows users to still see those around them, if they so choose.

THE POTENTIAL DOWNSIDES

My mixed feelings about Apple’s first foray into mixed reality ironically stems from just how well-designed the Vision Pro is by a company that has been behind this sort of game-changing technology on numerous occasions during the past 40 years, ranging from the Macintosh computer to the iPhone.

It feels like this may be another instance in which Apple has accomplished something that has eluded other tech companies by cracking the code to make both virtual- and augmented-reality more compelling and less disorienting than a variety of other ho-hum headsets have done over the past decade or so.

The only reason the Vision Pro isn’t going to be an immediate sensation is its cost. When it hits the U.S. market early next year, it will sell for $3,500, which makes it probable it will start out as a luxury item unaffordable to most households — especially because the headset isn’t going to supplant the need to buy a new iPhone or smartphone running on Android every few years.

The most likely scenario is that Vision Pro in some ways is Apple’s testbed for mixed reality that will encourage the development of more apps especially designed to take advantage of the technology. The next ripple effect will be an array of other products equipped with similarly compelling technology at lower price points that stand a better chance sucking more people — including children — into a realm that threatens to deepen screen addictions to the detriment of real-world interactions among humans.

Apple Vision Pro Hands-On: Far Better Than I Was Ready For

I experienced incredible fidelity, surprising video quality and a really smooth interface. Apple's first mixed-reality headset nails those, but lots of questions remain.

I went inside the Apple Vision Pro.

I was in a movie theater last December watching Avatar: The Way of Water in 3D, and I said to myself: "Wow, this is an immersive film I'd love to watch in next-gen VR." That's exactly what I experienced in Apple's Vision Pro headset, and yeah, it's amazing.

On Monday, I tried out the Vision Pro in a series of carefully picked demos during WWDC at Apple's Cupertino, California, headquarters. I've been using cutting-edge VR devices for years, and I found all sorts of augmented reality memories bubbling up in my brain. Apple's compact — but still not small — headset reminds me of an Apple-designed Meta Quest Pro. The fit of the back strap was comfy yet stretchy, with a dial to adjust the rear fit and a top strap for stability. The headset's sleek design, and even its glowing front faceplate, also gave me an instant Ready Player One vibe. 

I couldn't wear my glasses during the demo, though, and neither will you. Apple's headset does not support glasses, instead relying on Zeiss custom inserts to correct wearers' vision. Apple did manage, through a setup process, to easily find lenses that fit my vision well enough so that everything seemed crystal clear, which is not an easy task. Also, we adjusted the fit and tuned spatial audio for my head using an iPhone, a system that will be finessed when the headset is released in 2024.

From there, I did my demos seated, mostly, and found myself surprised from the start. The passthrough video camera quality of this headset is good — really, really good. Not as good as my own vision, but good enough that I could see the room well, see people in it with me, see my watch notifications easily on my wrist. The only headset that's done this previously was the extremely impressive but PC-connected Varjo XR-3, and Apple's display and cameras feel even better.

Apple's floating grid of apps appears when I press the top digital crown, which autocenters the home screen to wherever I'm looking. I set up eye tracking, which worked like on many other VR headsets I've used: I looked at glowing dots as musical notes played, and got a chime when it all worked.

A list of apps as they would appear inside of the Apple Vision Pro headset.

From there, the interface was surprisingly fluid. Looking at icons or interface options slightly enlarges them, or changes how bold they appear. Tapping with my fingers while looking at something opens an app. 

I've used tons of hand-tracking technology on headsets like the HoloLens 2 and the Meta Quest 2 and Pro, and usually there's a lot of hand motion required. Here, I could be really lazy. I pinched to open icons even while my hand was resting in my lap, and it worked. 

Scrolling involves pinching and pulling with my fingers; again, pretty easy to do. I resized windows by moving my hand to throw a window across the room or pin it closer to me. I opened multiple apps at once, including Safari, Messages and Photos. It was easy enough to scroll around, although sometimes my eye tracking needed a bit of extra concentration to pull off.

Apple's headset uses eye tracking constantly in its interface, something Meta's Quest Pro and even the PlayStation VR 2 don't do. That might be part of the reason for the external battery pack. The emphasis on eye tracking as a major part of the interface felt transformative, in a way I expected might be the case for VR and AR years ago. What I don't know is how it will feel in longer sessions.

I don't know how the Vision Pro will work with keyboards and trackpads, since I didn't get to demo the headset that way. It works with Apple's Magic Keyboard and Magic Trackpad, and Macs, but not with iPhone and iPad or Watch touchscreens — not now, at least.

Dialing in reality

I scrolled through some photos in Apple's preset photo album, plus a few 3D photos and video clips shot with the Vision Pro's 3D camera. All the images looked really crisp, and a panoramic photo that spread around me looked almost like it was a window on a landscape that extended just beyond the room I was in. 

Apple has volumetric 3D landscapes on the Vision Pro that are immersive backgrounds like 3D wallpaper, but looking at one really shows off how nice that Micro OLED display looks. A lake looked like it was rolling up to a rocky shore that ended right where the real coffee table was in front of me. 

Raising my hands to my face, I saw how the headset separates my hands from VR, a trick that's already in Apple's ARKit. It's a little rough around the edges but good enough. Similarly, there's a wild new trick where anyone else in the room can ghost into view if you look at them, a fuzzy halo with their real passthrough video image slowly materializing. It's meant to help create meaningful contact with people while wearing the headset. I wondered how you could turn that off or tune it to be less present, but it's a very new idea in mixed reality.

Apple's digital crown, a small dial borrowed from the Apple Watch, handles reality blend. I could turn the dial to slowly extend the 3D panorama until it surrounded me everywhere, or dial it back so it just emerged a little bit like a 3D window. 

Mixed reality in Apple's headset looks so casually impressive that I almost didn't appreciate how great it was. Again, I've seen mixed reality in VR headsets before (Varjo XR-3, Quest Pro), and I've understood its capabilities. Apple's execution of mixed reality felt much more immersive, rich and effortless on most fronts, with a field of view that felt expansive and rich. I can't to see more experiences in it.

Cinematic fidelity that wowed me

The cinema demo was what really shocked me, though. I played a 3D clip of Avatar: The Way of Water in-headset, on a screen in various viewing modes including a cinema. Apple's mixed-reality passthrough can also dim the rest of the world down a bit, in a way similar to how the Magic Leap 2 does with its AR. But the scenes of Way of Water sent little chills through me. It was vivid. This felt like a movie experience. I don't feel that way in other VR headsets.

Avatar: The Way of Water looked great in the Vision Pro.

Apple also demonstrated its Immersive Video format that's coming as an extension to Apple TV Plus. It's a 180-degree video format, similar to what I've seen before in concept, but with really strong resolution and video quality. A splash demo reel of Alicia Keys singing, Apple Sports events, documentary footage and more reeled off in front of me, a teaser of what's to come. One-eighty-degree video never appears quite as crisp to me as big-screen film content, but the sports clips I saw made me wonder how good virtual Jets games could be in the future. Things have come a long way.

Would I pay $3,499 for a head-worn cinema? No, but it's clearly one of this device's greatest unique strengths. The resolution and brightness of the display were surprising.

Convincing avatars (I mean, Personas)

Apple's Personas are 3D-scanned avatars generated by using the Vision Pro to scan your face, making a version of yourself that shows up in FaceTime chats if you want, or also on the outside of the Vision Pro's curved OLED display to show whether you're "present" or in an app. I didn't see how that outer display worked, but I had a FaceTime with someone in their Persona form, and it was good. Again, it looked surprisingly good.

I've chatted with Meta's ultra-realistic Codec Avatars, which aim for realistic representations of people in VR. Those are stunning, and I've also seen Meta's phone-scanned step-down version in an early form last year, where a talking head spoke to me in VR. Apple's Persona looked better than Meta's phone-scanned avatar, although a bit fuzzy around the edges, like a dream. The woman whose Persona was scanned appeared in her own window, not in a full-screen form. 

And I wondered how expressive the emotions are with the Vision Pro's scanning cameras. The Pro has an ability to scan jaw movement similar to the Quest Pro, and the Persona I chatted with was friendly and smiling. How would it look for someone I know, like my mom? Here, it was good enough that I forgot it was a scan.

We demoed a bit of Apple's Freeform app, where a collaboration window opened up while my Persona friend chatted in another window. 3D objects popped up in the Freeform app, a full home scan. It looked realistic enough.

Dinosaurs in my world

The final demo was an app experience called Encounter Dinosaurs, which reminded me of early VR app demos I had years ago: An experience emphasizing just the immersive "wow" factor of dinosaurs appearing in a 3D window that seemed to open up in the back wall of my demo room. Creatures that looked like carnotauruses slowly walked through the window and into my space. 

All my demos were seated except for this one, where I stood up and walked around a bit. This sounds like it wouldn't be an impressive demo, but again, the quality of the visuals and how they looked in relation to the room's passthrough video capture was what made it feel so great. As the dinosaur snapped at my hand, it felt pretty real. And so did a butterfly that danced through the room and tried to land on my extended finger.

I smiled. But even more so, I was impressed when I took off the headset. My own everyday vision wasn't that much sharper than what Apple's passthrough cameras provided. The gap between the two was closer than I would have expected, and it's what makes Apple's take on mixed reality in VR work so well.

Then there's the battery pack. There's a corded battery that's needed to power the headset, instead of a built-in battery like most others have. That meant I had to make sure to grab the battery pack as I started to move around, which is probably a reason why so many of Apple's demos were seated.

What about fitness and everything else?

Apple didn't emphasize fitness much at all, a surprise to me. VR is already a great platform for fitness, although no one's finessed headset design for fitness comfort. Maybe having that battery pack right now will limit movement in active games and experiences. Maybe Apple will announce more plans here later. The only taste I got of health and wellness was a one-minute micro meditation, which was similar to the one on the Apple Watch. It was pretty, and again a great showcase of the display quality, but I want more.

2024 is still a while away, and Apple's headset is priced way out of range for most people. And I have no idea how functional this current headset would feel if I were doing everyday work. But Apple did show off a display, and an interface, that are far better than I was ready for. If Apple can build on that, and the Vision Pro finds ways of expanding its mixed-reality capabilities, then who knows what else is possible?

This was just my fast-take reaction to a quick set of demos on one day in Cupertino. There are a lot more questions to come, but this first set of demos resonated with me. Apple showed what it can do, and we're not even at the headset's launch yet.

Apple Unveils Vision Pro: A Revolutionary Leap In Mixed Reality Technology

In a move that comes as no surprise to industry insiders, Apple has officially joined the race in immersive digital technology with the launch of its highly anticipated Vision Pro mixed reality headset. The tech giant is stepping into an already crowded field, competing against heavyweights like Microsoft’s Hololens, Meta’s Oculus, and Magic Leap. However, Apple is confident that its revolutionary device will stand out from the competition. Apple Vision Pro will retail for $3,499, and it will be available for purchase in early 2024.

Described as a “revolutionary spatial computer,” the Vision Pro headset aims to seamlessly merge digital content with the physical world, all while keeping users connected to others. Apple’s CEO, Tim Cook, emphasized the device’s groundbreaking capabilities, stating, “Vision Pro is years ahead and unlike anything created before – with a revolutionary new input system and thousands of groundbreaking innovations.”

One of the standout features of the Vision Pro headset is its control mechanism, which responds to the user’s eyes, hands, and voice inputs. Apple claims that the device can interact with digital content in a manner that feels physically present, projecting high-resolution imagery into the user’s physical space. This immersive experience is made possible by the headset’s 23 million pixels across two displays and the integration of a custom Apple silicon chip.

To complement the hardware, Apple has developed VisionOS, a spatial operating system that offers a three-dimensional interface, liberating applications from the constraints of traditional screens and integrating them into real-world spaces. Initially, users will have access to over 100 Apple Arcade games, but the company hints at the potential for new game experiences that can span a spectrum of immersion.

Addressing concerns about the intrusiveness of wearing a headset, Apple has implemented “EyeSight,” a feature that allows users to see others while also displaying the user’s eyes. This design choice aims to eliminate the “creepiness” often associated with wearing such devices.

While critics may argue that the Vision Pro headset borrows ideas from existing devices, its release carries significant weight in the development of immersive digital experiences. Apple’s App Store is already a highly lucrative platform for developers, and the company boasts an extensive and passionate customer base, with over two billion active devices. By venturing into the mixed reality space, Apple has the potential to shape the future of Web3 and redefine how we interact with digital content.

Interestingly, Apple’s marketing materials intentionally avoid terms like “virtual reality” or “metaverse.” This strategic move hints at the company’s desire to position the Vision Pro headset as a distinct and innovative technology, separate from existing categories. It signifies Apple’s commitment to pushing the boundaries of immersive digital experiences and highlights the potential for exciting new possibilities in this rapidly evolving field.

As the Vision Pro headset hits the market, the tech world eagerly awaits the response from consumers and developers alike. Apple’s entry into the mixed reality arena is undoubtedly a significant milestone that will influence the trajectory of immersive digital technology. With its blend of cutting-edge hardware, user-friendly interface, and vast ecosystem, Apple is poised to leave its mark on the immersive digital landscape, shaping the future of how we interact with the digital world.