Plays like an interactive movie as much as a video game.

The Quarry, a just-released video game from Supermassive Games and published by 2K Games, is described as an interactive drama/horror along the lines of a teen slasher flick like Friday the 13th, as nine teen counselors struggle to survive their last night at Hackett’s Quarry summer camp, while fending off strange creatures and locals run amok.

Who will survive and who won’t is up to the gameplay decisions of the player, who assumes control of all nine playable characters (in single-player mode). In fact, a lot rests on the player’s choices during the game—for example, whether the person opts to investigate a certain sound or instead run and hide, assist the others or take an “every person for themselves” stance. All decisions, both big and small, affect the game—from the character development to plot development. And, collectively, those decisions dictate how the game’s story ends. Will everyone survive? Will just some make it through the night? Or will all perish?

There are more than 180 possible endings and a script that exceeds 1,000 pages due to the branching story line. And there are about a dozen different ways in which a character can die. According to Supermassive Games, the title can last anywhere from seven to 10 hours, depending on the life span of the characters.

While The Quarry is indeed a game, it sounds an awful lot like an interactive movie. In that vein, its creators went the extra mile (several miles, in fact) to deliver it in Hollywood style, with realistic characters and authentic performances. Assisting the developer in that effort was Digital Domain, known for its cutting-edge VFX work, particularly in the area of photorealistic digital humans. But could the same workflow that is used to push boundaries in film also be used for a game? Well, pretty much.

According to Aruna Inversin, creative director and VFX supervisor at Digital Domain, the studio started with the same tools it uses for movies and episodics, then adapted them for game development. “Basically, we came up with a new, faster way to animate photorealistic digital characters,” he says.

Creating the cast of characters

The Quarry features an ensemble cast of some well-known actors, including David Arquette, Ted Raimi, Ariel Winter, and Justice Smith, among others. Because they are easily recognized, it was imperative that the digital characters look like the actors. For this data acquisition, they used DI4D’s 4D scanner.

Yet, the reasoning extended beyond recognition; there was the matter of authenticity. They also had to “act” like them as well. “It was important for us to make every movement and reaction as realistic as possible, to create an immersive experience where players truly feel the weight of their decisions and connect to the characters,” says Will Bules, game director for The Quarry. “We wanted to get as close to the likeness of the actors as possible, both in terms of the super-high-detail scans and the subtleties and nuances of their performances.”

To capture the performances, Digital Domain held an extensive motion-capture shoot spanning 42 days and resulting in 32 hours of footage, during which each actor donned a mocap suit along with a facial capture rig. For the body motion on stage, a Vicon system was used; for the facial capture, Technoprops helmets were employed. The actors then performed their roles, acting out all the different versions of the story line. That data was processed using the studio’s Masquerade 2.0 facial-capture system, which is now powered by machine learning. Of the 4,500 completed shots approved for the game, only 27 needed to be retouched by an artist, Digital Domain contends.

As for the facial tracking, the studio came up with ways to improve the quality of the results; they also developed a new tool for editing the CG characters in real time for the game platform. First, Digital Domain digitally scanned the faces of every cast member, which provided an accurate template to start the work and resulted in a library of facial shapes and expressions for each person. From that data, the artists at Supermassive Games crafted the overall look for their in-game characters.

Next, the CG game characters along with the performance data were fed through Masquerade, which combines the actor’s live-action footage with that person’s facial shapes from the library that was created. Then, a photorealistic CG version of the performance is generated using machine learning, resulting in a fully animated character performance in no time at all.

The authenticity of the performance, however, hinges on the talent of the actors, as every performance appearing in the game is an exact replica of each facial movement, emotive reaction, and nuance delivered by the actors. “We wanted to ensure that our tools amplified what the cast brought, so that each performance down to the subtlest movement is as true to the live-action performance as possible,” says Paul “Pizza” Planezza, senior producer at Digital Domain.

While the studio has used this technique to deliver a number of recent character performances in films, including that of Thanos in Avengers: Infinity War, this particular usage was not for a prerendered application like a movie. Rather, these game characters had to be edited in real time, without the usual post cleanup of the facial-capture data. To make that possible, however, Digital Domain first had to overcome two specific hurdles associated with facial capture that resulted in the need for the data cleanup: eye tracking and helmet stabilization.

Eye tracking is often affected by factors such as head pose, user distance, illumination variations, occlusions, and display properties of the setup, resulting in gaze estimation errors. To eliminate those errors, Digital Domain started with the open-source GazeML, which uses a deep learning framework based on TensorFlow for training high-performance gaze estimation. Then, over the course of three years, they heavily modified the program. According to the studio, by using machine learning to identify the uniqueness of a cast member’s eyes, and pairing it with a high frame rate head-mounted camera, GazeML was able to produce better tracking accuracy, which led to an improvement in the look of each character’s digital eyes.

During the facial tracking process, better results can be achieved when the actor keeps his or her head relatively stationary while wearing the headcam—not easy to do, however, when a performer is struggling for his or her virtual life during a multitude of scenes and scenarios. As a solution, Digital Domain devised a new proprietary technique whereby machine learning algorithms in the Masquerade 2.0 system analyzed the captured data and compensated for any jostling that occurred. That gave the cast the freedom to move around normally and unrestricted while wearing the head rig.

One additional machine learning technology was used for the game’s digital cast, to make the characters more authentic: Chatterbox. Chatterbox, which analyzes the facial expressions of each performer and determines the best option to use, was developed by the studio’s Digital Humans Group for its work on creating photorealistic autonomous digital humans, including the recent advancements for the autonomous CG Zoey (https://gfxspeak.com/2022/05/16/building-better-humans/). By using Chatterbox, a large amount of unnecessary data is eliminated, making it possible to upload the CG characters and edit them directly in a game engine—in this case, Epic’s Unreal Engine—so the game development team can see the character performances within the game’s environments and make real-time adjustments to those performances as needed. Since the options are all based on the actor’s own unique facial movements and mannerisms, the results retain the desired level of believability. Moreover, with Chatterbox, the game studio was able to avoid reshoots, which can be expensive.

The Quarry was released June 10, 2022, for Windows, PlayStation 4 and 5, Xbox One, and Xbox Series X/S.   

What do we think?

When all was said and done, Digital Domain says it rendered more than 250 million frames for The Quarry—far, far more than is done for a typical film. But this is not a typical film. It’s a game. Or is it an interactive film? What’s not in question is the extent to which the developer went to make the title so authentic. Game imagery is no longer constrained by interactivity, as it once was. And here is one impressive example of that. But back to the film/game question. Supermassive Games has been pushing the envelope with games that pay tribute to the horror and monster genres, such as Until Dawn (2015) and The Dark Pictures Anthology (2019–2021). For this title, they delved even further into that genre.

Helping The Quarry maintain a foothold in the game world and a foot, or at least a toe, in the interactive film world, was a visual and technical endeavor. The developer borrowed on various movie tropes from different eras to establish and bolster that dual presence. For instance, although the game is set in modern times, it also pays homage to popular horror films like those from the 1980s by giving the settings and characters a retro feel. Also, the game relies on the filmic convention of building up fear through growing tension—it does not rely on the twitch reaction of killing enemies en masse. And, recruiting well-known horror actors, such as David Arquette from the Scream film series and Linda Shaye (known as a scream queen from her roles in various horror films), to star virtually in the game further tied in the movie association.

Supermassive Games even expanded on the game as a cinematic experience by incorporating a movie mode in which players can alter the personalities of the characters and their various actions, and then watch the action unfold—ideal for lookie-loos. With so many branching story lines and possible endings, The Quarry will feel like an interactive movie for sure. Of course, there are plenty of conventions that situate this title within the gaming camp, including an online multiplayer mode, the ability to unlock gameplay advantages and more.

So, is it a game or an interactive movie? Decide for yourself. But one thing is certain, descriptions of certain forms of entertainment that were once definitive are now changing, thanks to so many technological advancements, which are resulting in hybrid forms of entertainment. So, call it a game or call it an interactive film. But definitely call it a “killer” endeavor.