Apple Vision Pro Could Bring VR Into The Mainstream

It has been seven years since Alejandro G. Iñárritu and Emmanuel Lubezki presented CARNE y ARENA at the Cannes Film Festival. In an airport hangar 20 minutes down the coast from the Palais, the pair had created a vast volume with a sand floor as participants strapped into a virtual reality headset and found themselves joining a group of immigrants on their perilous passage from Mexico into the United States. At the time, Deadline called the installation — which would transfer to LACMA among other venues — the first step in the birth of an artform.

In the years since, there have been precious few developments in virtual reality to suggest artists have truly capitalized on the potential of perhaps the most literally immersive form of storytelling imaginable. During the pandemic, Valve Software caused a run on VR headsets — including its own Valve Index — with the release of Half-Life: Alyx, a videogame that took its own giant steps forward for the medium. Tech companies including Meta and Sony went all-in on VR, or its hybrid virtual/real cousin, augmented reality. But there have only been scant examples of VR innovating as an artist’s medium.

With the February release of Vision Pro, Apple, the tech giant renowned for squaring the circle on nascent technologies that haven’t yet quite hit the mark, hopes that it has created a platform that may just change all that. Could Vision Pro be the paradigm shift that makes VR and AR an essential tool for artists and audiences? The possibilities are tantalizing.

Even tech reviewers who have seen dozens of headsets come and go have been impressed with the smart implementation of various technologies inside Vision Pro. From its intuitive user interface built around simple glances and gestures, to its lightning quick response times and high-resolution screens, the developers behind Vision Pro have worked hard to eliminate the many VR pain points that cause an “uncanny valley” effect that many users struggle to shake.

Apple’s unique approach to Vision Pro has been to attempt to define a new market segment. Vision Pro is not a headset that requires any kind of tethering to a Mac or PC, nor is it a low-powered remote device. In fact, it has the same computing hardware as its own upper echelon of MacBook Pros built right in. Its visionOS is adapted from its other, highly successful operating systems for its portable devices. Vision Pro is not exclusively a streaming service, a personal computer, or a videogame platform. Just as iPhone was introduced as an iPod, a phone and an internet communicator all at once, Vision Pro is intended to serve all of these functions simultaneously.

The device has already attracted the attention of filmmakers like Jon Favreau, who created some of the headset’s launch showcase films, and James Cameron, who said his experience was “religious”. But it is also priced at $3,500 in its most humble configuration, and Apple analysts suggest sales have been sluggish. Many tech reviewers have cautioned that Vision Pro, in its current form, may not be the device that capitalizes on its own potential, but that future iterations of the headset may well get there. Vision Pro, they say, must be seen to be believed.

So, what might the potential be for the creative industries?

Streaming

Perhaps the least revolutionary of Apple Vision Pro’s toolset is its capability as a streaming device for traditional 2D and 3D content. Native apps already exist for streamers like Apple TV+ and Disney+, while Netflix and others are available through the web browser.

It has long been possible to access streaming content through devices such as Meta’s Quest headset, the latest iteration of which starts at $499, and Sony’s PlayStation VR2, which retails at $549. But Vision Pro’s 23-megapixel micro-OLED display technology and beefed-up computing power does make streaming a much more pleasant experience. In concert with its augmented reality function, in which the device’s two screens — one dedicated to each eye — give users a view through the headset and into their own three-dimensional living rooms, throwing a cinema-sized projection onto the wall and watching as the ambient lighting from feature films and TV shows dance around the environment certainly makes Vision Pro feel more immersive than its competition.

Vision Pro also advances Apple’s implementation of Spatial Audio, with two small speakers hovering above the user’s ears delivering surprisingly powerful and effective virtualized 3D audio.

But the device weighs upwards of 600 grams (1.3 pounds) excluding its external battery pack, and that battery offers only 2-2.5 hours of semi-tethered power, meaning Vision Pro may not yet be a committed binger’s dream.

Filmmaking

With stereoscopic cameras built into the Vision Pro, as well as the company’s latest iterations of the iPhone, the 15 Pro and 15 Pro Max, users can capture awe-inspiring 3D home videos and photographs. Enterprising filmmakers are almost certainly going to be able to capitalize on these technologies for their own storytelling, and while the videos are still framed — that is to say, they won’t give a full 360-degree virtual experience — the hologram effect is breathtaking.

The real excitement for professionals, though, comes with Apple Immersive Video. The company has created a proprietary camera rig for capturing 180-degree 8K video content. One of the sample clips involves a train running along a snowy, mountainous track, and it seems like no accident that the sense of immediate immersion recalls the apocryphal stories of audiences running from the theater when they first experienced the Lumière brothers’ L’arrivée d’un train en gare de La Ciotat in 1895.

The Immersive Video camera rig is an Apple exclusive at present, but from a seated position, its potential is immediate. Want to experience a play without leaving the house? Apple Vision Pro could offer the best seat available, on demand, and the stage is no longer confined to a theater.

Many VR users report motion sickness when watching 3D immersive content that features a lot of movement. Apple has carefully curated its sample clips to keep the camera largely steady, or at least keep movement to a minimum. It will be up to filmmakers to process how to capture shots without making films that feel like stomach-churning rollercoaster rides.

Live Events

Apple Immersive Video’s installation at major sporting events and concerts is another tantalizing prospect. Demonstration footage of Major League Soccer games, with a 3D camera centered directly above the goalposts, goes a step beyond delivering just the best seat in the house. Indeed, watching sport from such a unique vantage point grants an entirely new perspective on the action.

Another example features a performance by Alicia Keys inside a rehearsal room. The placement of the camera once more puts viewers in the heart of the action, among Keys and her band. What if Apple Immersive Video came to the Oscars red carpet, or inside the Dolby Theatre itself? Suddenly Hollywood’s hottest ticket might not be quite as out of reach as it once was.

The real trick for Apple will be to keep the content coming, and to figure out the right ways to stream it live. With an 8K resolution at a high framerate of 60fps, and with such an expansive field of view, compression will be a key consideration. So far, Apple has showcased only short samples of what Immersive Video can do. No current longform TV shows or movies, nor live events, are available in the format, and early adopters have been disappointed by the slow drip of new footage. Apple will need to move fast to line up examples that make good on these demos.

Production

Perhaps the most intriguing innovation for filmmakers is less about audience delivery. As an augmented reality computing device, Vision Pro’s use on film sets suggests exciting opportunities for all departments.

Filmmakers on YouTube have already teased Vision Pro’s monitoring abilities. Video Village takes up valuable space on set, but Vision Pro understands where users are spatially. Cinema-sized screens monitoring camera output can be placed in any convenient spot and will stay there as users move away from them to complete other tasks.

Imagine a costume department in which reference images and videos are pinned to the walls behind designers, from which to readily refer as they work. A workspace designed for script supervisors to keep logs with little more than a chair to work from. Focus pullers may no longer require small reference monitors to keep shots sharp. The use-cases are multitudinous.

Filmmaking is a collaborative medium, and Vision Pro’s front-facing lenticular screen, which give a virtualized image of the user’s eye movements from the front of the headset, means the device doesn’t necessarily have to be removed for conversations to continue unhindered. The effect is not quite as pronounced as it appeared in Apple’s early teaser videos, with the device’s reflective front face making it more subtle than the company might have hoped, but it is a clever idea set for future enhancement. 

Battery life will also hinder potential set usage, especially in remote locations. But might a cache of backup batteries take up less space on an equipment truck than the many monitors, laptops, and other devices currently essential on a film set?

Apple’s gamble with Vision Pro is bold: to carve a new niche for a technology that had all the hallmarks of approaching obsolescence, like 3D cinema before it, when Apple first announced it. Despite a handful of promising implementations of VR and AR, Mark Zuckerberg’s vision of a future inside the metaverse still feels more meme than meaningful, and even Vision Pro’s heavyweight form, beautiful as it is to look at, struggles to overcome the fatigue of extended operation of a headset.

But the first iPhone launched in 2007 with 2G internet and no App Store; only the suite of applications Apple had designed for it. By the time its successor arrived 18 months later, that had all changed, and the smartphone era truly began. Those tech reviewers are right to imagine a fruitful second or third iteration on this concept. And if any company can sink the dollars necessary into slimming down the device and its price, it is Apple.

Its success, though, will rely on third parties; filmmakers and users willing to embrace its potential. Just as cinema was defined as much by its practitioners as it was by those that developed its technology, so will the wider industry and audiences be the ones charged with establishing the language of this new paradigm.

Apple Vision Pro does feel like the first device of its kind to make that journey truly worthwhile. Competitors will doubtless spring up to emulate some of this innovation now that Vision Pro is in the wild, but even at a price point nearer $500, its rivals have struggled to overcome the gimmicky nature of VR, and paucity of truly revolutionary content, to muster large sales volumes. Vision Pro is certainly more impressive than its rivals technologically. Only time will tell if Apple’s high-priced dice-roll will truly ignite the future of a new artistic medium.