NASA’s hybrid reality astronaut training system uses 3D scanners

3D scanning technology is becoming vital tool for space exploration missions.

NASA established Hybrid Reality Lab in late 2015 based on Unreal Engine 4 and incorporating consumer-grade VR, physical mockups and models, wearable technologies, and room-scale technology to create training environments. One of the applications of the lab is a detailed mixed reality (MR) replica of the International Space Station (ISS) for training new astronauts. It is a dynamic simulation of most of the interior modules frequented by American astronauts. Immersive environment technologies make the training experience as realistic as possible while running various simulations.

**NASA has built a virtual version of the out-the-window view from the International Space Station’s Cupola. (Source: Nvidia)**

The NASA installation uses a combination of real physical spaces as well as digital models to give astronauts the feel of being in the actual environment. In other words, the developers of the Reality Lab have built a physical stage to complement the digital model so people inside the VR world feel walls, ledges, etc. That’s why NASA calls this trainer hybrid reality.

NASA has implemented 3D scanning to create convincing virtual reality environments. It uses Artec Space Spider structured light 3D scanner to create virtual replicas of tools that astronauts may use in space. For example, Pistol Grip Tool, which is used for drilling bolts. X-Ray Fluorescence Tool used to determine soil/rock composition on Mars. The concept is particularly important to allow astronauts to experience how they will work in space since the absence of gravity is difficult to reproduce prior to launch.

**The real-world object (left) vs. its 3D scanned equivalent (right). (Source: Artec)**

A major goal is to simulate reduced gravity and the sense of tactile feedback. Presently, NASA’s Johnson Space Center operates the Active Response Gravity offload system (ARGOS).

“It is essentially a smart tether which attaches to your back, offloads your body weight and accounts for your momentum in the vertical and horizontal directions to make you feel like you are in Lunar gravity, Martian gravity, microgravity or anywhere in between,” said Matthew Noyes, Software Lead at NASA’s Hybrid Reality and Advanced Operational Concepts Lab.

The Hybrid Reality Lab environment has been merged with the ARGOS so that the user can move through a virtual representation of the International Space Station. Multiple users can train together in the same virtual reality environment, working together even if they’re not in the same physical location.

What do we think?

NASA is establishing hybrid reality as a new frontier of space exploration. It is using 3D scanning technology to create precise and realistic models of tools that are used in space or will be used on future missions. These tools then can be imported to a virtual environment for simulation training.

3D scanning and printing finds many uses in space exploration in the future. Be it scanning and printing of a broken part on the spacecraft or collecting research data by 3D scanning objects found in space.

CEO of Artec 3D, Artyom Yukhin says Artec scanners help control changes in the shapes of astronauts’ bodies, as those are precise and totally safe for scanning people. He doesn’t deny the possibility of scanning aliens if an astronaut happens to meet any!