Filmmakers Gary Yost and Adam Loften combine 3D camera footage with computer graphics.

At Siggraph 2021, filmmakers Gary Yost and Adam Loften showed their piece Inside Covid-19, which tells the story of one man’s battle with Covid-19 as a doctor and as a patient. The documentary follows Emergency Room supervisor Dr. Josiah Child who, early in the pandemic, developed safety protocols for hospitals in Northern California, and who had a very rough ride with the virus himself. Their documentary brings the virus into focus and makes it personal.

The Emmy-nominated VR documentary is a production of WisdomVR, a company founded by Yost in 2018 as a non-profit with support from Oculus (now Facebook Reality Labs). Yost has had a long and fascinating career in visual technology. He is well known for founding the Yost Group, who developed Autodesk’s 3D Studio with Dan Silva, Don Brittain, Rolf Berteig, and Tom Hudson with whom Yost worked on Atari projects in the mid-1980s. Yost is also an award-winning photographer and filmmaker.

His most recent projects with WisdomVR are efforts to capture and preserve stories of our culture and our collective wisdom. In all, the WisdomVR project has released 10 experiences that can be watched on Oculus TV.

Yost says he was drawn to VR and Oculus for his documentaries because he liked the idea of young people using Oculus encountering these documentaries about people and important subjects they may never have otherwise encountered. WisdomVR Project is creating a living library of immersive experiences and future plans call for the development of a free curriculum, enabling more people to create WisdomVR experiences within their communities to share worldwide.

Yost’s encounter with Josiah Child came through a mutual friend who had been encouraging him to meet Child as a possible subject. Their meeting had been delayed by the onset of Covid-19 and then because Dr. Child had contracted the virus. When Child and Yost finally met, Child was in the early stages of a long recovery from the virus.

Yost says he and Loften recognized that Child could offer insights into the Covid-19 experience from the point of view of a scientist and a patient. In a published quote, Loften says that he hoped their documentary could help people learn from the Covid experience. “We believe that if we can unite and create resilience at this moment, it will guide us through the next challenges inevitably faced by humanity.” Likewise, Yost says he hopes this documentary will function as a lasting reminder of our collective experience. He notes that somehow almost all of humanity has managed to forget the flu pandemic of 1918 and so we failed to respond to this one as seriously as we should have.

Two stories

There are actually two protagonists in the documentary. The first is Dr. Child, who, with his family, reenacts his experience with Covid as he was at his home in the mountains outside of Santa Fe, at the northern California hospitals where he worked, and the hospital bed where he fought the virus. The Covid-19 virus is the other major character in this piece and Yost draws on his experience with 3D visualization, working with longtime friend and animator Andy Murdock to show how the virus works and multiplies inside the human body.

There is a sequence in the documentary that shows the virus entering the body and replicating itself. It’s beautiful and also the stuff for horror movies. Yost says he wanted these sequences to be as realistic as possible. In a video for the Siggraph conference, Yost and Murdock describe how they created the alien universe of the virus. In our conversation, he also described how his team used Chaos V-Ray shaders for the render because V-Ray is very efficient at complexity. Phillip Miller of the Chaos Group donated >$30,000 of rendering time, which, says Yost, is the only way they could have produced that footage with the high quality that it has. Yost also noted that Nvidia’s Frank DeLise helped them out with the donation of high-powered Titan cards.

Yost worked with Whitney Lai of VR specialists Surgical Theater to use CT scans of Child’s lungs captured during his illness. Using the scans, Lai created voxel-based 3D models that graphically showed what is the pulmonary emboli and inflammation that happens to the lungs in a severe case of Covid-19.

The documentary takes advantage of the wide vistas of the high desert where Child lives and contrasts them with the claustrophobic confines of hospital beds and sickrooms. There are times the viewer may actually feel that it’s hard to breathe as we sit with Dr. Child in a hospital room or when we see the 3D scenes of Child’s lungs fighting against the hardening of tissues caused by the disease. Certainly, when we see Child out in the mountains around his home, we understand his appreciation for the fresh air around him.

The documentary has a very immediate feel, thanks in part to the very tight shots the team was able to get. Yost says that in the early days of WisdomVR, he went looking for tools to create 360-degree content. He had worked with GoPro and other 360 cameras and was unsatisfied with the results. Eventually, he started working with ZCam and their V1 camera, a 10-camera radial array that he said was able to give him the natural stereoscopic imagery he wanted.

There’s no looking away with VR. VR has been called an empathy machine. It is that and more. The encompassing, yet intimate VR view, enables the viewer to identify with what is being seen through the headset. It is an experience with the power to bring another level of understanding and possibly, imprint itself in memory. There are several studies that have found VR to be a valuable tool for learning and remembering. (To name two: a study published by Surface at the University of Syracuse found that VR helped people learn second languages, and another from the  University of Maryland that found VR helps people learn tasks easier.)

Yost is not done with the Covid-19 virus. The animations we see in Inside Covid-19 are Yost’s prototype for a further project he’s working on. The Wisdom VR team is looking for seed funding to create a deeper look into the pandemic and an interactive exploration of the immune system. Yost is planning this new project with biomedical animator Drew Berry from the Walter and Eliza Hall Institute of Medical Research in Melbourne, Australia. His work has focused on creating 3D and 4D animations to help explain cellular and molecular processes. Biomedical researcher Gaël McGill, founder of Digizyme and developer of Molecular Maya, is also on the team. McGill is Director of Molecular Visualization at the Center for Molecular and Cellular Dynamics at Harvard Medical school.

There are a lot more stories to tell and there are many ways to tell them. VR is coming into its own as a medium for telling stories even as it’s becoming evident, once again, that most people really don’t want to wear a headset to play games. Instead, VR is currently a short form capable of communicating a great deal of information in a few shots.

What do we think?

Inside Covid-19 is a relatively long VR video at 37 minutes, yet I was fascinated by the piece and didn’t really notice the length of time. With this documentary, Yost and Loften have helped push the boundaries of the medium on several fronts.