The company’s latest AI-powered developments for artists.

As Adobe prepares to celebrate its 40th anniversary at the end of the year, Shantanu Narayen, Adobe chairman and chief executive officer, couldn’t resist using the stage at Adobe Max 2022 to highlight some major Adobe accomplishments over the years, from the invention of desktop publishing to revolutionizing imaging and design, to pioneering electronic documents, to advancing animation, gaming, and video. In 2016, Adobe launched Sensei, its artificial intelligence (AI) and machine learning (ML) framework, which proved to be another industry game changer.

Adobe has been strengthening the products and capabilities across its ecosystem using the power of Sensei. That, as well as tighter product integration and better collaboration, were the main threads at this year’s Adobe Max conference last week.

David Wadhwani, president of Adobe digital media business, reminded the audience that we are currently living at a time when content, creativity, and design have never been more valued, when content is fueling the global economy, when digital content consumption is exploding, and when virtually every business needs a digital presence. And that creates a plethora of opportunities for artists. But it also creates challenges, such as keeping up with demands on creatives’ time and having less bandwidth to experiment and learn new things. Adobe’s goal, he said, is to help take what is in an artist’s mind and enable that person to create it with higher production value and in less time.

This is where Adobe Sensei comes in. Over the past few years, Adobe has been investing more and more in Sensei, the company’s AI engine. Wadhwani pointed out that Adobe customers already are using many features powered by Sensei to automate mundane tasks—and Adobe continues to work on new capabilities that can take their core flagship application to new levels. However, with Sensei, it’s not all about productivity gains; Sensei-powered features also enable more creative possibilities by giving artists more time to explore and iterate.

“Our goal is to make AI your copilot in all of your creative endeavors,” said Narayen.

Adobe is using Sensei to free artists from mundane tasks and deliver new speech-to-text, for example, in Premiere Pro, which is using AI to automatically generate captions for videos with a single click. And, liquid mode in Acrobat uses Sensei AI and ML in the background to identify parts of a PDF document to make them easier to read on mobile devices. Last year, Photoshop and Lightroom AI image-editing features were used 1.3 billion times for a range of mundane tasks such as removing blemishes on a portrait, selecting detailed objects, and more, according to Adobe.

“Imagine, if you can, combining generative tech with Lightroom so you can ask Sensei to transform night into day, a sunny photograph into a beautiful sunset, move shadows, or change the weather,” says Wadhwani. “This is all possible today with the latest advances in generative technology; generative AI can conjure up an image simply from a text description.”

During Max 2022, Adobe announced new features that just launched in its flagship Photoshop application that are bolstered by Sensei AI. These include:

• Selection improvements that enable users to make detailed selections of complex objects with a single click, creating more accurate selections of elements such as skies, foregrounds, subjects, and hair, while preserving detailed edges.
• One-click delete and fill selects and removes objects from images, filling the removed area with content-aware fill, in a single action in Photoshop on the Web—as if the object was never there.
• Photo restoration neural filter (beta) uses machine learning so users of Photoshop on the Web can eliminate scratches or other minor imperfections on old photos.
• Remove background is now available in Photoshop on the Web (beta), enabling one-click background removal by isolating the main subject (person, animal, vehicle, object) from the background.
• Masking and brushing for making precise adjustments faster and easier from a Web browser are also now available in Photoshop on the Web (beta).

Additional new AI capabilities were added to Adobe’s Creative Cloud apps and Express, the company’s all-in-one tool that gives users access to the functionality of Adobe’s flagship creative tools, including Photoshop and Illustrator, in an online and mobile design app.

New to Adobe Express:

• Remove Background saves time by using AI to detect and highlight the key subject of a photo, so a new image can be placed into the background, chosen from a variety of designs and destinations.
• Recommended templates, one of the most popular features, allows anyone to quickly get started creating social media content, logos, and other projects using thousands of professionally designed templates. Adobe’s contextually aware AI recommends a customized collection of relevant templates.
• Font recommendations leverages contextual AI understanding of project content to identify and recommend appropriate fonts from a collection of over 20,000 Adobe Font typefaces. This is available now on desktop and mobile devices.
• Refine cutout, now available on mobile devices, uses AI to identify additions to or subtractions from already-selected parts of an image, making it easy to remove unwanted elements.

AI features have also been added to Lightroom for desktop, mobile, and Web. These tools use AI to improve specific parts of an image, such as the sky, background, and people; there are also many new Lightroom adaptive presets that improve the whole image, not just selected portions, with a single click.

New to Adobe Lightroom:

• Masking selection enhancements for precise selective adjustments.
• Select objects lets users select objects using different methods, including roughly scribbling on the photo to choose an object or dragging a selection box over the object while Lightroom AI automatically refines the selection.
• Adaptive presets are single-click enhancements unique to a photo that can quickly enhance an entire portrait, or target specific features with presets including eye enhancement, tooth whitening, eyebrow darkening, and more via a single click or tap.
• Content-aware remove makes edits to images.

Insofar as video is concerned, Adobe has been working to make editing and production quicker and easier through AI and other innovations. Auto Color, a one-click color-correction tool powered by Sensei machine learning, enables non-expert colorists to jump-start on the color-grading process using Premiere. Turning to music, Premiere Pro’s remixing tool now does the heavy lifting when it comes to fades.

New to Adobe Premiere Pro and After Effects:

• Premiere Pro auto color applies intelligent color adjustments and accelerates color correction workflows.
• Remix automatically retimes music to match the selected video clip.
• After Effects scene edit detection uses AI to automatically detect scene changes in an edited video clip, and then turns scenes into individual layers or creates markers at edit points for faster project setup.

New to Adobe Substance:

• Adobe Substance 3D AI-enhanced materials build on Substance 3D Capture’s existing 3D materials workflows with new AI functionality that can create and intelligently enhance photorealistic 3D textures from a live camera or specified images. This enables the creation of 3D content based on any physical location.

Adobe is continuing to add AI capabilities to its offerings, and there’s more to come in the next phase of AI features, which will help generate assets for users. “Imagine being able to ask your creative copilot in Photoshop to add an object to a scene simply by describing what you want or asking your copilot to give you an alternative idea based on what you’ve already built,” Wadhwani said.

Social responsibility and AI

Despite the advantages that AI and ML provide, Narayen is aware of the darker possibilities of generative AI if the technology is misused, and Adobe is assuming responsibility by ensuring its technology is used to make a positive impact. So, through its Content Authenticity Initiative (CAI), Adobe is partnering with over 800 organizations globally to combat misinformation and to provide attribution and provenance for creatives’ work. This not only credits the creator, but also shows from whom the image originated—something that is becoming increasingly needed in an era of misinformation.

Last year, Adobe introduced Content Credentials in beta that embeds attribution information into photos edited in Photoshop, to combat image disinformation, and this year the company is further improving on this and making the feature available to all Creative Cloud subscribers in Photoshop. Now, two leading camera manufacturers, Nikon and Leica, are implementing content authenticity into their two new cameras, the Z9 and M11 Rangefinder, respectively, making it easy for photographers to attach content credentials to their work at the time a shot is taken.

Content Credentials goes a long way toward preventing fraud, from photos to NFT authentication. Content Credentials will be able to show what was created by a human and what was created by a machine. And as generative AI becomes a bigger part of the creative process, this information will be increasingly essential.

Wadhwani further promised that Adobe is approaching generative technology from a creator-centric perspective, with the belief that AI should benefit human creators, not replace them. “We want you to be excited about the technology we are  building and how we go about building it,” said Wadhwani.