The company demo’d the virtual production technology currently being used in car ads that live up to the tradition of high-gloss paint jobs and unnatural nature shots and promise a future of digital twins.

Unreal is staking out more ground in the Metaverse as they see it during their Unreal Build Automotive Event, which, of course, this year was virtual. Unreal has been pitching the automotive industry with Build conferences in Munich and Detroit. The Munich events have been held in 2018, 2019, and in 2021, this virtual event brings in the world to see what can be done with game engines.

The Epic Unreal Enterprise group is finding the automotive industry to be promising territory for the work Unreal has been doing in virtual production. Creating advertising and marketing content is the obvious on-ramp. Even before the pandemic changed the way we interacted with sales and services, the car industry has been interested in virtual product techniques because it allows vendors to develop commercials and glossy ads before the car rolls off the production line; while it’s still a prototype. In one of the earliest instances of virtual production for car commercials, Unreal collaborated with the Mill and Chevrolet in 2017 to create the Blackbird rig, a moving target if you will for car commercials.

They produced a video called The Human Race to unveil the 2017 Chevrolet Camaro ZL1 as it races with an FNR autonomous concept car, killing two birds with one stone: here’s the car you can have right now, and here’s the car we may sell you … later. The only car filmed, in the process of making the shot, was the Mill Blackbird. It’s a car rig that’s fully adjustable in size and shape and allows car bodies to be mapped on it for driving scenes.

In his introduction to the 2021 virtual event, Simon Jones, Director of Unreal Engine Enterprise, talked about how far techniques have come now. Unreal is busy building the Metaverse, a digital version of the world which will include as a matter of course digital twins for the automotive industry, as well as digital buildings, digital cities, digital people, all made up of 3D models, materials, lights, geography, a digital representation of everything.

The Automotive event was a follow-on to Unreal’s Build event in 2019 held in Munich before lockdowns changed everything.  To demonstrate how the automotive industry was taking advantage of digital technologies to market cars, Doug Wolff, Business Developer Manager, Unreal Engine Enterprise, described BMW’s pivot to virtual technology to “Live Launch” their new electric prototype vehicle. Stefan Ponika of BMW said they realized that they’d have to find a new way to launch BMW #NEXTGen 2020, which they had originally envisioned as live events to introduce the BMW iX, which BMW positions as an Electric Crossover because of its 300-mile range and 500 HP.

Instead, the company created a series of splashy videos using an LED Cave and under the direction of Elastique creative agency. And, instead of trying to recreate the typical car commercial driving through dramatic landscapes, the group played with the environments including fanciful twinkling forests, sci-fi skyways, and a pastel design environment more typical of an architectural design model than any place that exists in the world. The content creators also edited together shots in which it seemed people and cars were actually in the environments and then pulled back and showed them on stage. These works celebrated artifice; they didn’t hide it.

Hyper shooting

The Hyperbowl studio was opened in Munich in July 2020. The studio, which provides over 400 square meters of LED screens, was a collaboration between ACHT studio advertising agency, NSYNK Studio, TFN, and Fournell Showtechnik. In addition to all the LED real estate, the Hyperbowl offers 20.32 m × 14.32 m of floor space to accommodate events as well as studio shoots. The volume includes 258 square meters of LED sky. It is currently the largest virtual production space in Europe.

Volkswagen used the space for the Volkswagen ID 4 EV rollout and flashy is the keyword or would be if neon weren’t the other keyword. Frank Foerster, Hyperbowl’s GM and CEO of TFN, said, “you can’t shoot a car commercial without an environment.” Cars are all about the reflections, and glossy surfaces, he added. For the Volkswagen shoot, they created multiple environments. What would have been a very complicated shoot, requiring closing streets and moving production from site to site, was actually done in two days, thanks to virtual production.

Hyperbowl’s Creative Director Julian Krüger said that when he came to Hyperbowl he realized how dramatically some of the work done in post-production had gone to pre-viz. He said that using the Hyperbowl meant that the environments had to be created first and brought into Unreal to be sure that they could be rendered in real-time. Once they had that part figured out they could then get to work and move smoothly from one scene to the other and between dramatically different backgrounds.

The Hyperbowl is also getting a lot of use these days for virtual keynotes and live events. They can combine real-world props with digital, have AR elements, and work with concept cars.

HMI

One of Unreal’s newer products for their automotive initiative is their HMI Initiative which is being used for developing and configuring automotive interfaces in the car. It can be used to design the real thing for new cars, for car UIs in games, or for fantasy machines in movies and television. Unreal is offering an HMI starter pack, with UI templates.

Unreal is adding support for embedded OSs such as Blackberry QNX Nutrino, Android, and Linux. They’re also integrating Mapbox for navigation systems. The HMI design tools are available in the Unreal Marketplace.

In automotive, Unreal is working closely with General Motors on the development of HMI design tools, and the Hummer EV will be the first to use Unreal Engine in its digital cockpit. The story is the Hummer will be using Unreal Engine to power its infotainment display. The interface is receptive to sensor data and can communicate real-time information about the car. “Unreal is with you all the way,” so the slogan goes.

Scott Martin, the creative director for GM’s User Experience and User Interface design, says GM is working closely with GM and that the Unreal team has helped GM to learn how to better use the limited memory in-car systems to get the best effect. The heightened visuals, he says, allows their customers “to engage with their vehicles in new ways.”

Stephen Gray, head of GM’s Global Creative Visualization (GCV) group, says they create the first public view of a new GM product. He has worked with Maya Vred, Adobe products, Nuke as their primary tools, and now they’re adding on Unreal Engine. Traditionally, GM has had several different pipelines for content creation, which is kind of an of course statement if you’re thinking of making commercials, designing cars, and developing HMIs. But, Gray says, as a result of incorporating Unreal Engine into their workflows, they’re finding points at which they can combine workflows. They’re finding places where they’re comfortable using Unreal in final production. He says, “our customers don’t care how a shot is created, only that it looks great.”

Gray says they’re still using live shoots for video content in cases where the verisimilitude is a valuable component. Remember, these are the guys who led the development of the Blackbird. Gray says the immersive team has found that the use of real video and live photography adds a lot to create the suspension of disbelief, but they are also combining the live work with Quixel Megascans and photogrammetry said Gray talking about possible future directions GM may take for visualization. “I can’t say more, now,” he teased.

Stephen Murphy, GM’s Strategist IT Immersive, says Unreal is being used by GM in design, production, engineering, IT, and marketing. He says the teams all have an Unreal project in the works. They use Unreal and VR in many areas to improve processes and they’re looking at putting Unreal to work visualizing math data.

Car Configurators were some of the first tools to be part of the digital car design continuum—they cross comfortably over from games to car design allowing designers to try out HMIs.

Italian automotive company Pagani is actively working to knit the worlds together. Pagani builds dream cars to order. Customers design their own cars specifying all elements of the design. Customers work with Pagani’s online configurator to design their cars. Their latest model is the Huayra.

Pagani teamed with Porsche’s design consultancy MHP, and Mackievision to build a modern configurator that is able to present customers with a very convincing ray-traced car. Mackievision lead product manager Emma Schröder says Pagani customers are building personal works of art. There is no Pagani showroom, so the company needed to create an application that would make customers comfortable with the idea of buying a car without being able to slam the doors and kick the tires. Pagani’s cars cost millions of dollars and, and only a few models are actually built. Most are designed for tracks and private collections.

Founder Horacio Pagan headed Lamborghini’s composites design group and went on to found Pagani Composites and continued to develop materials for Lamborghini. In 1992, he founded Pagani Automobili to build his own cars.

The Pagani configurator is an integrated point of sales tool for customers and for Pagani sales. When customers settle on their car, the configurator can generate hundreds of images for quality assurance. The configurator relies on Nvidia RTX ray tracing and real-time 3D streaming from Monkey way.

Pagani also uses the images created by the configurator to create marketing materials, brochures, and ads. MHP went a step further taking a Huayra out to the track and creating a high-quality configurable film of the car going through the switches and turns of a German raceway. Customers can interactively change the colors of the car in the video as it plays. As the car changes colors, the lighting on the car conforms to the actual environment it is in. Videos created can be downloaded.

Metaverse: the bigger picture

Unreal’s Metaverse is a long-running work in progress. The acquisitions the company has made including Quixel, Datasmith, 3Lateral, Cubic Motion, Capturing Reality, and more are giving it the pieces it needs to see the Metaverse through all the way to true digital twins, or at least that is the aspiration.

That’s a whole, whole lot of pixels the company is talking about.

As Unreal prepares to launch Unreal Engine 5, it is showcasing the capabilities it is developing for different industries as well as games.

Coming up in Unreal Engine 5 are a number of features from Epic’s acquisitions and new partnerships and they promise to push the technology further. The company is promising virtual characters with its MetaHuman Creator tools that have come out of the company’s acquisition of 3Lateral, Cubic Motion, and Quixel. Quixel already plays a major role for automotive tools, offering a slate of materials specified by the automotive companies including materials from PBR (physically based rendering), materials optimized for ray tracing, and textiles for interiors.

Nanite, a major advance for the Unreal Engine 5 supports high-quality 3D models from any source, says Epic, including CAD, Zbrush, and Photogrammetry and enables high-end visualization through the new Lumen lighting system. For the cherry on top, Unreal has announced a partnership with Cesium as a plugin that will deliver 3D geospatial data for Unreal users—meaning developers can recreate any place on the “mapped surface of the Earth.” Unreal’s new photogrammetry tool, Reality Capture, can also generate objects and geography from photographs.

It all sounds pretty magical now, and from what the company showed at its Automotive conference, it does have magical talents. But, nothing brings a beautiful vision down to earth like CAD. Thomas Convard, Senior Product Manager for Epic Games, said that Epic’s Datasmith technology eases the passage of CAD models into the Unreal Engine. In addition, Epic has acquired the Twinmotion technology that enables rendering for BIM and AEC models. It has one-click synchronization for ArchiCAD, Revit, Rhino, and SketchUp Pro. Unreal’s Datasmith tool enables the manual transport of CAD into Unreal, which is one example of how CAD and content creation tools can come together.

The automotive companies have signaled they’re on board. Before the Pagani configurator, Audi and Mackevision created a digital showroom using Unreal Engine 4 and the companies said they worked with Audi’s CAD content to create models for the configurator.

For the most part, the models being used in the 3D volumes are prototypes developed with 3D content creation tools such as Maya, but Epic has clearly stated that they are moving to support digital twins from CAD so they can go to work in advertising, movies, games, and even with configurators and service tools. Can the data go back and forth? That would fulfill the demands of digital twins, and that’s still a big—if not insurmountable—hurdle for Unreal, which is optimized for real-time efficiency not real-world complexity.

It’s a crazy big vision, with some significant gaps. But, a look back to the first demos of the work done by the Mill, Unreal, and GM shows how far all these teams have come in just a few short years and a pandemic lockdown—which has certainly contributed focus and a sense of purpose.

The world is changing way too fast to rule out any player committed to true digital twins.

What do we think?

Siemens Digital Industries may have more of the puzzle pieces for digital twins within its enormous portfolio than its competitors in automotive manufacturing. When it talks about digital twins, it says there are multiple digital twins for all the systems involved in an automobile and yet the company is still focused on providing its customers with one version of the truth.

Epic would probably agree. Its work in HMI as exemplified in the Hummer is an example of a digital twin subsystem. Its work is reflecting the steps the major CAD companies are taking towards digital twins. In fact, its HMI work has replaced Dassault’s RTT technology, which is now Dassault’s 3DExcite. The entertainment content creation world and the CAD design world used to be universes apart. That’s no longer true. The evolution of HMI tools is an example of how the two sides of visualization and engineering are working to put the puzzle pieces together. Partnerships that were once unlikely could become more common as they try to digitize and connect complex manufacturing and aftermarket systems.

Keep an eye on Autodesk’s relationship with Unity to see how this is taking shape in AEC and also Nvidia’s fluid alliances as it seeks to build its version of the Metaverse, the Omniverse.