Ukraine’s Zelenskyy uses high tech to reach out to big tech leaders.

What better way to communicate with an important group on an important matter than by speaking in a language common to all? That is the tack Ukraine’s beleaguered President Volodymyr Zelenskyy took very recently in his speech to investors and corporate leaders. He delivered his speech in English and Ukrainian, but it was also delivered using complex technology most in the room would understand/appreciate. And you might say that Zelenskyy technically did not deliver the speech; it was his hologram that did so.

The Ukraine president, through his holographic self, addressed Europe’s top tech entrepreneurs, investors, and corporate leaders last week via a 3D volumetric video during the invite-only Founders Forum London and three other large European tech festivals. He pleaded with these “big tech” innovators to donate much-needed financial and technological resources to Ukraine, in an effort to rebuild his country whose infrastructure continues to be targeted by Russian troops.

This unique holographic presentation, especially given the logistical challenges of a war zone, was created using technology from New York-based Evercoast, which provided the volumetric capture, rendering, hosting, and streaming; the branded Web AR player; and all associated logistics and planning to execute the volumetric production in two cities: London and Kyiv. Evercoast was introduced to the project by Talesmith, the production company on the project.

As Zelenskyy stated, Ukraine continues to develop even now, and promised to make digital transformation central to his country’s rebuilding. The hologram helped underscore that commitment with actions as well as words. “We live in a time of super opportunities. Your technologies, knowledge, expertise, and investments—all of these can show their value and do it in Ukraine. No other country in the world will give you such a chance to use the most advanced technologies at the state level,” he implored the audience. “Ukraine is a chance for a global digital revolution.”

Behind the curtain

In real life, this may be the first time a country’s leader used a hologram to deliver an important speech, or anything else for that matter. And the historical aspect was not lost on Ben Nunez, cofounder and CEO of Evercoast. “[We] had the extraordinary opportunity to produce and deliver a seminal moment in volumetric video with the holographic capture of President Zelenskyy. This is an incredibly important message from a leader whose country is under siege,” he says.

For obvious reasons, the project was mired in secrecy. Now, however, Nunez is finally able to discuss the project, in which time was of the essence.

The idea of creating Zelenskky as a hologram originated with individuals from Founders Forum, Martin Williams of Talesmith, and Thomas Hoegh of Garden Studios in London. A Ukrainian-born entrepreneur based in London was also instrumental in bringing the idea to fruition. Common colleagues had introduced Evercoast to the Talesmith team, which included the director, Williams, and producer, Mark Wright, both of whom organized the primary logistics and led the creative direction for the content. Nunez recalls their first chat, during which Williams and Wright cryptically mentioned a project in a high-risk part of the world. Early discussions moved to an encrypted voice and chat platform where they could talk more freely and communicate the details of the project.

“We had our first call with them on June 1, and were on a plane to London on June 6,” Nunez says.

The production occurred on location in Kyiv and at Garden Studios in London.

Needless to say, the small group did not know for sure what to expect when they arrived in Kyiv. After all, this was not a typical volume capture shoot, and many hurdles stood in their way. First, they had to ship a volumetric capture system on very short notice across the world at a time when Ukrainian airspace was locked down. Also, Nunez and Williams had to get to Kyiv, essentially a war zone, themselves—a logistical nightmare starting with securing the proper medical and liability insurance, crossing the border, and traversing the country—all the while trying to remain safe. Once in Kyiv, they had to locate a local grip and gaffer to set up the proper lighting; finding lights and other production gear in Kyiv was no easy task, either, as so much had been moved outside the heavily bombarded city. On top of that, they had to work with the president’s team to coordinate with his schedule and remain on a ready-state, and once in the presidential palace, where the production would take place, deal with the unfathomable security.

And of course, secrecy, again, was of utmost importance.

According to Nunez, there was no set time allotted for preproduction, since they had no idea what they were getting into while in Ukraine in terms of available resources, the space provided, the Internet speed, Zelenskyy’s availability and schedule, and any logistical support. “It was an extremely fluid situation that seemed to change hourly. In many ways, we were flying by the seat of our pants,” Nunez says.

While a typical setup ranges from a half-day to a day, this was no ordinary situation, and it took two days to set up the system and be in a ready-state for the capture process.

The postproduction is fully automated and cloud-based, says Nunez. So once the system was up and running, capturing and rendering content and then making it available in the mobile AR experience that was also created “was as simple as clicking a few buttons,” according to Nunez.

Nunez describes the capture process used, starting with a 5 m × 5 m cube space, inside of which are lots of LED panel lights and eight poles configured in a 3m diameter circle. Twenty-one Intel RealSense depth cameras were distributed across those eight poles (and one was mounted overhead to get tops of heads, shoulders, and so forth). All of those cameras plugged in to a single high-powered workstation running Evercoast software, which is really where all of the magic happens, says Nunez. Evercoast’s Mavericks software captures depth and color from every camera, reconstructs it in 3D in real time for livestreaming, renders preview assets immediately after a take to review the performance, and then uploads and launches renders of the source data on the company’s cloud rendering platform, Cloudbreak. That is where the firm generates the highest resolution output, encodes and compresses the data, and then makes it available for streaming—all in a fully automated fashion and executed within an hour.

“This is an entirely proprietary software stack we’ve built from scratch,” Nunez says.

The production itself took just 30 minutes. “The president nailed his speech. He did two takes in English and two takes in Ukrainian. Both were great,” Nunez says. The end result was then delivered within an hour of the video capture.

The shoot took place on June 15, 2022; The Founders Forum was held June 14–16.

“The logistics and security were unprecedented. Nothing like this has ever been done, capturing a sitting president and doing so in a time of war,” says Nunez. “Pulling this off was a feat.”

In addition to the Founders Forum, the holographic video of the president’s speech was also shown simultaneously at three other of Europe’s largest tech festivals—Stockholm’s Brilliant Minds, Amsterdam’s The Next Web, and Paris’ Vivatech—and direct to smartphone users via the AR experience. It was also livestreamed to the Dublin Tech Summit, the AI Summit at London Tech Week, and SuperReturn Berlin.

A fresh approach

The attack on Ukraine began in February, and since then, Zelenskyy has pleaded for assistance from other world leaders, individuals… anyone and everyone, to help his country stand up to the aggression and the atrocities Ukraine is experiencing. He and his wife have held press conferences and numerous telephone and Zoom calls, all of which are bound to not have the same impact they once did. Indeed, Ukraine still needs funding and armaments, but it is now deploying a very valuable homegrown weapon. Zelenskyy is a former actor who knows intimately the power of presence. He and his team, including his press secretary, are very media savvy. They know the importance of staying front and center to retain people’s attention.

Volumetric video of a presidential message during war, distributed as a mobile AR experience, is unique and allows anyone around the world with a smartphone to “bring it close to home” by placing the president in their environment, notes Nunez.

Furthermore, Ukraine has had a strong reputation for engineering and technology talent. And in his AR speech, he’s focused not just on defending Ukraine, but also on rebuilding the country. “They’re determined to not allow this war to destroy the country and everything they’ve built over the past decade-plus,” Nunez points out. “In the speech, he’s appealing to the broader tech community to help this rebuilding effort and offering a plan for digital transformation.”

Heeding Zelenskyy’s plea, Evercoast donated the entire system used for the capture, and even trained local personnel how to use it—including the depth cameras, a supercomputer, all the cabling, and the calibration board—which they left in the hands of the president’s media team. “They understand the importance of advanced technology like this as both a new form of communication and content creation, as well as an education tool for the engineering community inside Ukraine,” says Nunez.

Let’s face it, the world is evolving from 2D to 3D. The way people consume content and communicate with one another will continue to become more immersive and dimensional. Yet, most of the tools that currently exist for 3D content creation of a person generate output that’s either cartoon/emoji-like or are just static scans that are then animated artificially. While cute sometimes, those approaches lack realism and often break the feeling of presence, says Nunez. “Volumetric video solves that problem. It captures you exactly as you are—the clothes you are wearing, your hair, eyes, skin—it’s you. The president’s message, if delivered in 3D as a cartoon, just wouldn’t have worked. It had to be a volumetric video,” he adds.

Hologram Zelenskky may not be a one-time event, as Nunez tickles our curiosity by saying there’s much more to come with this and from the president. Beyond the mobile AR experience, the use of volumetric video in virtual production will continue to help differentiate messaging, he notes. To this end, simultaneously in London, another Evercoast system was set up to capture celebrities and tech leaders like Ellie Goulding, Eric Schmidt, Karlie Kloss, Peter Gabriel, Nile Rodgers, and many others delivering a message of support for Ukraine. Those captures will be integrated into a 3D experience delivered in the near future, he adds.

What do we think?

Zelenskyy, a comic actor by trade, is not your average world leader on so many levels. But he has proven himself very adept at handling unfathomable circumstances and has emerged as a great leader. Through it all, though, he has not lost his wits—nor his wit.

Listening to Zelenskyy address the tech leaders and urging them to assist his country, one could not help but recall that scene in Star Wars, first in Episode IV: A New Hope and then brought back in Episode VIII: The Last Jedi (2017) of Princess Leia’s hologram pleading for help. Bolstering that image was the fact that during the volumetric video capture, Zelenskyy wore a Star Wars T-shirt and even added a reference to the film series, telling audiences that “[The hologram] is not the only element of Star Wars that we will put into place; we will beat the Empire, too.” A not-so-subtle reference to Russia. Well done, President Zelenskyy.