Can you interact with a salesperson or a product as if you were in the same room, even though you are on your computer, someplace else?

Volumetric storytelling is what we are talking about. It’s okay to admit you never it called that before. This involves rendering a 3D world that one can interact with via the computer, but it must be delivered to the user. Either take them into the world through the viewing goggle (virtual reality) or show them the image as it would appear in their home (augmented reality).

This technology is still in the “proof of concept” stage, with some tentative efforts at marketing. Perhaps one good example of volumetric storytelling in action was the holographic appearances of Ukrainian president Volodymir Zelensky at various European diplomatic gatherings, or ABBA’s recent virtual concert in London.

“I would define volumetric storytelling as deployed in AR or VR and leveraging 360-degree photography to produce a 3D model made through volumetric capture or motion capture.” Explained D.J. Smith, COO of VR and AR platform The Glimpse Group.

But how do you get there?

Stacking visual building blocks

First, you must start with the 360-degree capture of an object to render into 3D. This will involve placing the subject in the center of an array of cameras to capture the subject from all angles. In the case of the virtual ABBA concert, this took 160 cameras to capture the four band members.

“Volumetric capture is not cheap and is not lite,” said Courtney Harding, founder and CEO of VR and AR agency Friends with Holograms. The technique generates large-sized files that can choke on delivery over narrow bandwidths. If a volumetric file is delivered over a 5G network, that’s fine, but anything less will result in a “laggy” playback, she explained.

A 360-degree photo shoot has its own challenges to be worked out. How many people? Are they moving or standing still?  Are they speaking? Dancing? Juggling? One person — or four? “You have to capture all the data, all the people.” Harding said. And the video must be done in one take. If that scene takes 15 minutes, and someone sneezes in the last 30 seconds, you are stuck. There is no “cutting away” to another scene, so you may have to do it over again, she said.

How to deliver the goods

The volumetric story needs a platform to be seen. That means virtual reality (VR) or augmented reality (AR). Each method has its strengths and weaknesses, as well as its own sensibilities.

The 360 technology offers “great visibility into an environment, but it’s not comfortable in a VR headset,” said Smith,

AR provides a “unique experience, but isolated use cases and greatly hampered by the requirement to use phones.” He said. “Opportunities will greatly open with AR wearables.” Here Google and Apple are readying new plans for AR-friendly smart glasses, a concept Google tried once before.

As for virtual reality, it has “amazing and transformative experiential potential, however, [it is] hindered by early stage, bulky, expensive hardware and lack of great content in a wide variety of genres.” Smith noted.

Even the platform on which you plan to display a volumetric file will dictate the limits of your storytelling. The smartphone offers a small screen for an AR story. VR is bigger, offering a much wider field of view, Harding explained, but there will be a greater level of complexity.

“AR deployment is more subject-based,” Smith said. Think of showing a product in a customer’s living room, viewed through their smartphone. “Virtual reality is more environment-based. You are transported into the volumetric scene.” Smith said. “VR is an empathy machine. In the VR experience, the user has a sense of presence — you feel as if you are there.”

Showing rather than just telling the story

It’s that sense of presence that is the main attraction of volumetric storytelling. Shlomi Ron, CEO of the Visual Storytelling Institute, likened it to the science fiction experience of being teleported to another world. When executed correctly, volumetric storytelling should provide a “3D experience, letting the people interact” with the subject, be it a person or a product.

It is possible to replicate a real-life setting, allowing users to teleconference into a virtual sitting room, Ron explained. It would look like a real room. Users would be able to see everything and walk around in it.

Such a technology would be ideal for travel brands, “creating a sense of ‘being there’” when selling a destination vacation, Ron explained. The same technology can also be used for urban planning, product development and remote training.

Still, a creative would have to bring a different sensibility into the room when crafting a 3D shoot. “Think about it as game development,” Ron said. “You have to make sure that any potential angle and perspective has some value from the perspective of the user.”

Still figuring it all out

The problem with new technologies is that potential can be demonstrated, but practical use is harder to show. There is no checklist of sensible practices a digital marketer can “check off” when constructing a campaign using volumetric storytelling.

Harding encouraged people put on a headset “and get a sense of what is possible”. One such piece of content is “On the Morning You Wake to the End of the World”, which Harding cited as an example of volumetric storytelling. The short film recounts a day when people in Hawaii woke up to a smartphone alert about incoming ICBMs.

“Understand what good storytelling is, and what is possible,” Harding said. Then it will be easier to figure out what is good for the brand.

“It’s up to the marketer to apply the teleporting experience to the goal,” said Ron.

“The best thing digital marketers can do is to keep a close eye on industry deployments to understand the potential and start experimenting with small proof of concept activations.” Smith said.

Marketers should make a start on discerning strengths and weaknesses of the volumetric technique. They will have to develop their own checklist of best practices the hard way — by trying it.