As immersive video content becomes a prominent way to tell visual stories, talented visual artists like Elad Offer not only face greater challenges, but they have fresh new possibilities to deliver something the world has never seen before. One such example is Bono: Stories of Surrender, a new film available in both 2D and 3D versions, marking the first feature-length Apple Immersive Video film for the Apple Vision Pro headset. Offer was the film’s Creative Director, tasked with making both 2D and immersive versions the best they can be for two very different mediums.
Creating the First Feature-Length Film in Apple Immersive Video
Bono: Stories of Surrender is a unique project because not only is it the first feature-length Apple Immersive Video for Vision Pro, but it is also available in a traditional 2D form. The film also premiered last month at the 78th Cannes Film Festival in 2D and is available on Apple TV+ for non-Vision Pro owners in 2D as well. The two mediums are highly different, which meant different challenges for Offer.
“The hardest part was figuring out how to incorporate Bono’s art and style into an immersive experience that would help underline and amplify Bono’s performance and delivery without overwhelming it,” Offer tells PetaPixel. “How to, as Bono called it, bring the ‘giddiness’ that was part of his book and the stage show into 3D space in and around the cinema version and add the fully immersive sections that were shot on the special Apple high resolution, high speed, cameras.”
Details are still relatively scarce concerning Apple’s special cameras it uses for filming immersive content for Vision Pro, but it’s a safe bet that the rig is similar to this one from Blackmagic. Offer says that the cameras shoot at an “extremely high resolution” at up to 90 frames per second with a full 180-degree field of view.
Visualizing 3D in a 2D Workflow
Offer says that a particular technical challenge was trying to visualize the immersive version of the film in a traditional video workflow, which relies heavily upon 2D storyboards and software.
“We had to cheat perspective and composition quite a bit just to convey the right idea,” Offer explains. “The reality is that for some parts that just wasn’t possible and we had to do full 3D stereo compositions in order to really understand how it all works together in a 3D immersive space. The team at The-Artery, the creative studio responsible for all aspects from creative direction through execution, design, animation, and VFX, really had to constantly come up with new ways to help me convey the feeling and experience that I wanted the final product to give.”
This type of challenge carries over to trying to convey the immersive aspects of Bono: Stories of Surrender on Apple Vision Pro to people who don’t have the headset. It’s like trying to pitch an idea to someone in a language they don’t understand.
“It freaking sucks! I can talk for an hour and build a mountain of words to describe what it looks like but until you experience it in the Apple Vision Pro and really get hit with the impact of the medium, the feeling like you’re right there in the middle of it and the experience of the combination we created of building Bono’s art into an architecture of emotion that you as the viewer and the 2D movie are surrounded by, you just can’t really feel the emotion that comes from it,” Offer laments.
However, there is a silver lining.
“The upside is watching people go through it for the first time and seeing their floored reactions. Seeing experienced journalists left literally speechless is super gratifying.”
It’s especially satisfying to see these reactions because creating an immersive film takes a lot of work.
“In many ways working in the immersive VR space is more like theater than cinema. Sure, you still get to control what’s in front of the viewer but you have to put a lot more thought into what creates a pleasing composition when your brain takes into account the spatial relationships,” Offer says.
“Things that can seem quite messy as a 2D projection of the immersive scene, can actually feel very clean once your brain understands that one line is far behind the other line. We had to do a lot of the composition with 3D glasses on or adjust things after seeing them in the immersive view because what looked like a nice composition in a 2D window just didn’t look right when embedded in the 3D world.”
How Immersive 3D Video Differs From Traditional 2D Movies
With such an expansive space in which to play, composition can be quite challenging. In traditional 2D films with less field of view, filmmakers can use these limitations to more easily control where the viewer looks at specific times. Everything is more controlled and predictable.
With immersive content on Vision Pro, it’s hard to know precisely where someone might be looking at any given time.
“It is much harder in this medium to direct attention because your field of view is so much larger than a movie theater,” Offer explains. “Things are happening below and above you, so it takes a lot of restraint in order to keep the viewer looking where you want them to.”
Another aspect is that, unless editors specifically tweak something, nearly everything is always in focus, so using depth of field to control the viewer’s attention is also not necessarily an option.
“Many times less is more was the approach we had to take in order to make sure the attention stays where we wanted it to be rather than distracting the viewer. We also did things like moving the bright or colorful things at the center of attention slowly and preemptively towards where I wanted the attention to go next. Music and sound also play a large part and thankfully due to the Apple Vision Pro’s advanced audio that fully surrounds the viewer, you can use those to direct the attention in pretty much any direction. These are elements similar to a traditional 2D movie but on a much larger canvas, so at times they take more consideration,” Offer says.
While Niche and Challenging, Immersive Video Has Serious Storytelling Potential
So far, the critical response to Bono: Stories of Surrender has been very positive, both for the 2D and Immersive Video versions. However, Apple Vision Pro remains a niche, expensive technology that is not nearly as accessible as having a traditional display and an Apple TV+ subscription. As Offer describes, immersive video and the Vision Pro are still in an “experimental stage.”
“It’s quite exciting seeing how each filmmaker working in the medium uses it in new ways. I have no doubt that many traditional filmmakers will be tempted to try their hand at working in this new medium of visual storytelling,” Offer says.
“It presents so many new ways to create emotion through visuals and I’m very interested to see what comes next. In short, your guess is as good as mine but it’s a really cool moment to be part of! As for me personally, I’ll absolutely use the Vision Pro again. For the right project, there is really nothing like the opportunities to create emotions that the Apple Vision Pro provides and I hope to be using it again!”
There is another aspect of the Apple Vision Pro that Offer calls out. Because the Apple Vision Pro is the only way to watch Bono: Stories of Surrender in Apple Immersive Video, it is a tightly controlled environment. Everyone who views the immersive version of the film will see it on the same device with the same screens.
While it was challenging to match the color space and grading from one version to another because the Vision Pro’s micro-OLED screens can achieve perfect black levels, the team was able to approach the edit differently while still working to maintain a consistent look across all formats.
There were also technical hurdles to overcome, as this is the first feature-length immersive film for the Apple Vision Pro.
“We also had to consider how long of an immersive piece was even possible to play in the Apple Vision Pro at that extremely high bit rate without running into buffering and other issues; and helping beta test the Immersive Viewing Utility which allowed us to preview the work on the Apple Vision Pro – these are all different questions we were working to answer,” Offer says.
It’s always special to be the first across the line with a novel technology. While it can be difficult, Offer believes in the immersive video format.
“I do believe that immersive video will keep growing in popularity because as a medium it can really create such a strong emotional, visceral response that is really nearly impossible to match in a 2D movie. Especially on a device with such high resolution and frame rate like the Apple Vision Pro — your brain forgets what you’re looking at isn’t real.”
Bono: Stories of Surrender is available to watch now on Apple TV+ and in immersive video on Apple Vision Pro. The film was directed by Andrew Dominik with cinematography by Erik Messerschmidt. The film was produced by Apple Original Films, Plan B Entertainment, and RadicalMedia.
Image credits: Apple