Let the machine do it: Adobe leverages AI for reformatting

Adobe reframe for Premiere lets users automatically change the format of their videos. AI and machine learning help accelerate the process.

Adobe at IBC focused on the capabilities of content-aware fill quite a bit—not that the technology is new, but that it can do so much. This demonstration showed that it’s possible to remove the orange ball from all video frames of the game. It works, but it makes people look pretty excited about nothing. (Source: JPR)

At IBC 2019, Adobe’s motto was “get to the finish line faster”; so that includes plenty of new stuff Adobe has added to the workflow over the years and almost none of it is sexy. The point of this year’s pitch was that taking the advances in Premiere Pro together, Adobe has helped make editors’ lives easier. This includes the ability to use proxies with instant transcoding, the ability to work with and see clips in different formats and aspect ratios, content-aware fill, automatic reframing, and creating voice sync’d animations.

How to use Auto Reframe in Premiere Pro | Adobe Creative Cloud.

At IBC, Adobe showed off a range of cool stuff, but I was surprised at how modest the introductions were. The jack-in-the-box didn’t open to unleash a laundry list of new features and capabilities for Adobe’s video programs. Instead, Adobe focused primarily on productivity. As has been the case for several years now, Sensei figured heavily in the conversation and so did content-aware fill because it’s at the heart of quite a few time-savers such as line removal, reformatting, and all kinds of clean up.

The headliner was Reframe for Adobe Pro and actually it is kind of huge and IBC is a perfect place to show it off to an audience of appreciative content creators. Debuted at Adobe Max as Project Smooth Operator, Adobe reframe for Premiere lets users automatically change the format of their videos to square, vertical, or 16 × 9. The software automatically prioritizes people in the scenes. It uses AI and machine learning to accelerate the process. Users can add information to improve the outcome such as telling the software that there is a lot of motion in the scene. Titles and motion graphics are dealt with separately so they are positioned correctly in the new format and likewise, if there have been previous edits to clips, they are respected as the content is reformatted.

Video has changed a lot due to social media. People who make videos may often need to produce content in different and incompatible formats. One of the most surprising unintended consequences of social media is that vertical video is no longer an excuse to deride the inexperienced mobile user. Instead, vertical is becoming a preferred format for social media sites like Instagram.

These days, Reframe is a feature that cuts across all markets of video content creators. It saves us all from the bad results of automatic pan and scan (i.e., wide screenshot of two people arguing across a wide table is cropped to the table centerpiece) but also enables all the citizen broadcasters out there to repurpose their content quickly.

There is also a subtle message in that “get to the finish line faster” message. The focus is on giving end-users the ability to get their work done better and more efficiently and not eliminating work or jobs. And, that is the way it’s playing out. We are seeing social media create more jobs than artificial intelligence is taking away.

What do we think?

With a subscription, companies have been able to change the way they handle major platform shifts. They don’t do them. Instead, they can fix what needs to be fixed, see what customers are using and make it better, and maybe just kind of ignore what customers are not using.

It’s been 9 years since Adobe introduced its subscription plan. Measured by the response, it was successful in a “we will burn down your house and build you a new one” kind of way. Many customers went along with the deal, but they didn’t feel they had much of an alternative. By now, though, many customers and companies like the idea that they have the most up to date software and write it into budgets, and it doesn’t hurt that Adobe makes sure people get goodies along with their subscription. Fonts, mobile apps, new tools, regular updates.

Now subscription has fundamentally changed the way the company operates, and this phenomenon is going to spread across the industry. Fewer big product family transitions and more target releases, Beta tools, and experiments. Best of all, though, is that companies are able to forget closer relationships with their customers.