Atomic Fiction recreates the famous stunt in The Walk with Faceware.

On a warm summer day in 1974, French high-wire artist and stuntman extraordinaire Philippe Petit surprised morning commuters in lower Manhattan when a cable appeared connecting the two towers of the World Trade Center. Petit and his team put it there so he could do his high-wire act on it. The stunt lasted 45 minutes, and put Petit into the history books and the New York City jail.

The 2015 film The Walk tells Petit’s story in the most realistic way possible, combining live action with state-of-the-art special effect. The World Trade Center towers are gone, and New York City looks different that it did in 1974. Recreating such scenes has become commonplace in cinema. But you can’t fake walking a wire. Actor Joseph Gordon-Levitt plays Petit, and when watching the film you see him several times waking a high wire. Or is he?

First steps

The vast majority of shots in The Walk required some form of digital enhancement – some 672 out of 826 in total. The Oakland, California-based Atomic Fiction got the majority of the CG work, delivering 250 of those challenging shots under the direction of visual effects supervisor and studio co-founder Kevin Baillie.

Atomic Fiction was tasked with creating a fully 3D version of New York City, accurate to 1974 as viewed from the top of the World Trade Center. The studio also needed to create a fully digital double of Gordon-Levitt as Petit to integrate into these shots. In order to really sell the concept of the famous actor performing one of history’s most famed high-wire stunts, advanced face replacements technology was required.

“It was really important to Zemeckis that we always maintained the illusion of Joseph performing the stunts himself,” says Rudy Grossman, CG supervisor at Atomic Fiction. “Several shots, which recreated Philippe Petit’s real-life stunts, could only be performed by an experienced master high-wire artists. To achieve this, we had to replace the face of the stunt performer with the performance of Joe’s acting.”

The merging of face onto stuntman was required in 32 shots. “We had to take the footage of Jade Kindar-Martin, a former Cirque du Soleil performer turned stuntman, and overlay the face and performance of Gordon-Levitt,” explains Grossman. Atomic Fiction chose Faceware for the detailed work.

“We’ve been using Faceware ever since working on Boardwalk Empire in 2011, and it was absolutely key to achieving the visual fidelity you see in The Walk’s facial performances today,” says Grossman. “Faceware’s performance capture technology gave us a solid foundation for staying true to the essence of the actors’ performances and timing – it was invaluable.”

Facing reality

“Because The Walk is so deeply rooted in reality, there’s no room to hide the details of visual effects in fantasy, or in the imagination of what a non-human creature might look like,” explains Grossman, reflecting on the challenges inherent in The Walk. “Everybody is unconsciously an expert on the human face – we spend most of our lives talking to each other, looking at each other’s faces, and looking at our own.

“On top of that the actor, Joseph Gordon-Levitt, has one of the most well-known and recognisable faces in the world,” he continues. “That was our challenge. We needed to convince an audience full of experts that our computer generated face was just the same as the face they already know.”

Along with this already considerable challenge, Atomic Fiction had to work under another demanding proposition: 672 shots might not seem much in a world of 2,000-shot VFX epics, but the lower count was due to The Walk’s much longer takes – an artistic choice designed to emphasise the reality and immersion of the big event.

In order to deliver the necessary face replacement results, Atomic Fiction’s process began on the last day of shooting The Walk.

On set, the team equipped Gordon-Levitt with Faceware’s Pro HD Headcam, and had the actor perform each of the takes that would require facial replacement. He did so while walking along a strip of tape stretched across the ground, upon which he mimed the act of wire walking without the added risk.

“We probably captured somewhere in the range of 30 minutes’ worth of data with the Pro HD Headcams, and processed what editorial felt was best suited to the face replacements,” explains Grossman.

“The process was really simple: Joe just put on the helmet, we hit record using Faceware Analyzer, and he acted,” he continues. “We applied a few ‘feature lines’ and makeup markers to his face in order to serve as reference to our animators, but the Faceware software didn’t really need them.”

Once Gordon-Levitt’s facial performance had been captured, it was time to transpose the facial data onto the performance of stuntman Kindar-Martin using Faceware’s Retargeter.

“We would evaluate the performance of the face rig on the stunt double, and then adjust the animation as necessary, using the facial motion data captured with Analyzer as our foundation, and the helmet camera video footage as our reference,” Grossman says. “Once the animation was approved, we ran out a few initial render tests in lighting, which informed any necessary fine tuning to the face rig.”

Using this captured and retargeted footage, Atomic Fiction could place Gordon-Levitt’s face on Kindar-Martin’s body, creating the illusion that he was every bit as physically capable and experienced, without ever having performed the wire-walking actions himself.

A successful stunt

“Using Faceware completely removes the guesswork,” says Grossman. “The actor has provided the performance which the director has already chosen – as such, we know what the director wants, and we know exactly what the actor performed. So the time spent finding the right performance is instead spent making sure the digital face looks exactly as it should, and ensuring it contains the essence of the actor’s performance.”

Elements such as pupil movement greatly helped the animators when working on the face replacements, as they could offset it to create the exact eye line that the VFX supervisor was looking for. “The animators would also often heavily rely on the initial width and height range for the mouth and brow movement, and then layer in animation from there,” explains Grossman.

Such advantages allowed the Atomic Fiction team to focus on pushing the realism of the facial replacements to the extreme; they could rely on Faceware to work on the core processes, and then have the artists target their energies on the final polish, perfecting the more nuanced details of the performance.

“Before Faceware, the face motion capture process for feature film had traditionally involved a huge investment of personnel, hardware/software development, and capture space that could only be afforded by very large studios,” concludes Grossman. “Faceware takes all this and makes it possible for small and medium-sized studios like Atomic Fiction to work on lean-budgeted films such as The Walk.