Adobe’s 3D & Immersive team reveals product updates and plans for the future.
Adobe’s Substance team is an integral part of Adobe’s 3D strategy, or more correctly, the Substance team is Adobe’s 3D & Immersive group. Sebastien Deguy, formerly the CEO of Allegorithmic, is an Adobe Vice President and leads the 3D & Immersive group.
For that reason, Substance’s annual presentation of its tool updates has been even more anticipated than usual. People are curious to learn more about Adobe’s 3D strategy.
To catch up, Substance is a node-based tool that enables the procedural creation of materials. Users start from base materials and then create the material they want by adding or adjusting qualities using nodes to create a graph that defines the material. They can be further edited using sliders. The materials are tileable and can be used to create or enhance the surface of 3D models or 2D images. Substance materials can be used with 3D modeling and animation tools and in-game engines. Substance includes several modules including Designer for material creation, Painter for 3D and 2D painting, Alchemist for image-based textures, and the giant Source library of materials.
With this annual release, the 3D team released a long list up updates including support for Pantone Colors, improvements to the Python scripting capabilities with documentation, and new additions to the library of premade scripts. Substance Painter can export the original topology or as tessellated meshes. Graphics board support now includes the newest Nvidia RTX AIBs for AI Denoising, speeding up baking processes, and also Alchemist’s image-based material creation. Some of the popular ease of use features include the ability to hide geometry to work with otherwise hidden elements, and the ability to delete nodes in a graph. And, in the category of something for everybody Source has been updated with over 3400 new materials including materials created in-house and signature collections from guest contributors and new high-end optimized parametric materials.
The list is long, but for a better and more detailed overview, watch the Substance team’s presentation.
Above the surface, materials as modeling tools
As always, the company highlighted the work of its customers with the annual Insanity Awards. The Insanity Awards recognize Substance users who push the software and create amazing textures and materials that are works of art in themselves. The work Substance users do often enable materials to do the heavy lifting for 3D in a scene. The Insanity Awards include entrants that create detailed landscapes, stylized imagery, and detailed surfaces with depth, texture, reflectance, color.
For instance, Daniel Thiger creates the details of a crown on top of a basic crown shape. All the detail and depth is provided by the materials.
Oliver Lau has been experimenting with hair and fur over the years after he decided doing it in a 3D program like Maya was too hard. His hair and fur tools create a wealth of options for every kind of hair from cottony to realistic. This year, Lau has added feathers to the features in his products.
In this case, the tools enable any kind of feather desire from fluff to flamingo.
In a Substance Magazine article called “Blurring the lines between modeling and texturing,” the writer says that the Substance team has seen their users use height and normal maps in Substance Designer to avoid sculpting painful details onto the model. In answer, the Substance developers recognized an opportunity to take that work a step further and provide features to help the community “actually shape” their model through texture data rather than sculpt directly on the mesh. The work these artists are doing provides clues to how Adobe is thinking about 3D for its customers and how the Substance development team are feeding off the work being done by their customers.
At the heart of Adobe’s 3D strategy is the belief they can transform the process of 3D creation and make it more accessible for artists. Artists who model can create their own models but perhaps the process can be simplified. Otherwise, Adobe is providing 3D models in Adobe Stock that artists can use as a starting point to create their work.
Adobe hopes to turn 3D content creation on its head by giving artists a starting point that’s further down the pipeline to their desired result rather than facing them with a blank plane or environment. The 2021.1 update presentation did not provide much in the way of specific details, but there were plenty of ideas about what Adobe is up to.
Maybe one of the biggest hints was the announcement that Guido Quaroni has joined the Adobe 3D & Immersive team. At Pixar, Quaroni was the VP of R&D and led the development of the USD format. The USD format is the last in a long line of possibilities creative communities have worked with to create a universal 3D format that will enable artists to work across development platforms from companies such as Autodesk, Foundry, and SideFX to achieve a more iterative workflow. Nvidia has been developing its Omniverse 3D creation environment on USD and Apple has made USDZ the open format derived from USD for AR as its native format for its AR development.
Quaroni has been named Senior Director of Engineering on the Adobe 3D & Immersive team.
Looking ahead
In 2016, Oculus introduced Medium, a free 3D creation tool that enables model creation with digital clay in VR. Adobe took over Medium in 2019, and then we heard nothing more. The tool enables 3D modeling with digital clay in VR. During their presentation Adobe 3D reminded the audience of Medium and said, it’s available to users on Oculus Rift S, Oculus Quest and Quest 2, and HTC Vive. Basically, the Adobe 3D team brought up Medium to let their audience know that it is still an approach to 3D that the company is interested in developing, but they don’t have anything concrete to announce, yet.
Other work in progress by the team includes work on Physics Whiz which was revealed at Sneaks at Adobe Max 2020. They’re studying ways to put physics to work so that objects behave correctly. As an example, it would be easier to “drop” several books on a table so that they lie naturally than it is to try and place the books realistically in a 3D environment.
In another example, they talked about Scantastic, another Sneaks reveal in 2020. Scantastic is a tool that makes the process of scanning a real-world object and creating a 3D model easier. Adobe’s tool makes use of the mobile device in conjunction with their server-based photogrammetry pipeline. They say it’s a tool people can use to create models for Adobe Dimension and Aero.
If you’ve ever tried using the phone to scan an object, it’s not as easy as promised. A nice magical piece of software to help would be wonderful but again, Adobe warns, it’s early days.
Differential material graphs for procedural material capture
Sebastien Deguy shared a project that was developed by Adobe and MIT and delivered as a 2020 Siggraph paper. Deguy notes that his history with the technology goes back to his early Ph.D. research. Deguy started out asking about capturing material from a photograph and he says, “it’s a very hairy mathematical problem.” He put it aside as he continued his research work. Today, he says, advances in AI have come along to contribute solutions to the challenge.
What they’re doing is taking an image and letting AI search for a similar graph. What’s interesting is the graph the AI comes up with may look nothing like the sample image provided. The algorithm under development by Adobe then takes the image and “dives into the graph” to try and reproduce the image. The results shown during the Substance 2021.1 livestream were convincing. Deguy emphasizes that the end result isn’t an image-based material that is limited in the ways it can be modified. It’s a fully procedural model that can be taken in a multitude of directions to create new materials. Deguy says that Adobe Source is a valuable resource for this project since they have so many graphs that can be used to train the AI. He also noted this project is an example of the way Adobe research can act as a force multiplier for the Substance team’s developmental work.
And what’s up next? More magical software. The Adobe research labs are also tinkering around with parametric modeling. This looks like traditional CAD modeling married to the procedural mindset of the Substance team.
The idea is that, given standard shapes, users can assemble a model and start iterating. If users can take base materials and play, shift, and iterate to create something completely different, why can’t they do that with basic shapes, and then they can slap a detailed material on it and boom, done—3D content creation. This work rides on the shoulders of work that has gone before. Constructive Solid Geometry (CSG) is creating models by combining primitives, but Adobe is envisioning a large library of options and multiple parameters that can be changed to create new models. Again, we’re warned not to get too excited but again, these ideas take us a lot closer to more spontaneous 3D content creation than we’ve been before.
This evolution of 3D in Adobe demonstrates what can happen when people address a traditional technology from an artist-driven point of view rather than that of an engineer. That is not to slight engineers, but to highlight the different expectations the artist brings to a model as raw material. Clearly, the categories of artist and engineer are not mutually exclusive.
What kinds of 3D tools Adobe ultimately creates may be completely different from the hints we’ve gotten so far. Packaging can make a big difference. After all, Substance is a marvelous tool, and loads of the materials are free, or almost freely, available. The development tools are more expensive, and not delivered as part of a Creative Cloud package. And, as always is true in real life, they’re not as easy to use as they look. Let’s hope that our future is a world where everything is easier. But if everything isn’t easier, it might be nice for 3D content creation to be.
What do we think?
The important point here, and I think it is one Adobe would make explicitly, is that there are all kinds of ways to get to 3D and Adobe wants to open up access to the different routes.
The creation of art is never easy. Talent and creativity are always going to be a barrier, but it sure would help if the tools were not as big a barrier as they are.
Adobe has taken several runs at 3D, but what has gotten out in front of the public has always seemed like the company is nibbling at the edges.
Somewhere along the way, though, the lightbulb went off within Adobe. They’ve built teams of next-generation 3D specialists in product development and R&D who are taking advantage of technology advances in processing, rendering, AI, and 3D to rethink creative tool building. Even before the arrival of the Substance team, Adobe was developing 3D tools like Dimension, Aero, and Fuse that let people jump in and play rather than facing the steep learning curve of modeling.
Quaroni has said in an interview for Adobe’s blog that when Adobe acquired Allegorithmic, he was convinced they were serious about developing 3D tools.
After this presentation, everyone should be convinced not only that Adobe is serious about developing 3D tools but they’re serious about addressing 3D in a much more updated way.
