Amazon Web Services has developed new AI-powered tech meant to help TV networks and Hollywood studios more effectively compete in the vertical video space.
AWS on Tuesday launched AWS Elemental Inference, a service that transforms live and on-demand video into vertical formats in real-time, allowing for live vertical streaming of sports, news, and other types of programming with minimal latency (AWS says it only expects a 6 to 10 second delay).
AWS expects media companies to use the tech to stream on social platforms like TikTok and Instagram, where vertical video rules, as well as to create vertical formats for use on their own streaming platforms. Fox Sports and NBCUniversal have already signed on to use the tech.
“By processing content during encoding rather than post-production, broadcasters can publish social-optimized clips and highlights while live events are still in progress—turning speed into a competitive advantage for audience engagement,” explains Regina Rossi, head of product for AWS Media Services.
“For vertical video creation specifically, we use fully managed foundation models optimized for video workloads, including capabilities for saliency mapping, object tracking, and speaker detection,” Rossi adds. “These models analyze each frame to identify subjects, track movement across scenes, and intelligently determine the right vertical placement — keeping athletes centered during plays, speakers in frame during interviews, and key action visible regardless of aspect ratio. We apply smoothing algorithms so the viewer experience feels natural, similar to a cameraman panning across a scene.”
Major streaming platforms have experimented with vertical video (Netflix, Peacock and ESPN all have some actively available right now), but the process for converting conventionally shot content into vertical content is a bit more complex than it may appear at first glance. Live programming like sports is even more complicated. AWS is betting that its tech will make it vastly easier, enabling more vertical video on streaming platforms to reach consumers when they aren’t at home, and on social platforms where they are spending a lot of their time.
“In speaking with our customers, they have use cases for both distribution channels,” Rossi says. “Putting live vertical streams and clips on social platforms helps with content discovery and captures viral moments in real time — reaching audiences where they’re already engaged. Having more vertical content in their existing apps also provides viewers with an experience that feels more mobile-friendly. Fox Sports identified that nearly 90 percent of their digital content is consumed vertically, and this is a growing expectation for viewers across all mobile platforms and apps.”
“Automatic vertical conversion is essential because the vast majority of Fox Sports Digital’s audience consumes content on mobile, within platforms built for vertical viewing,” added Ricardo Perez-Selsky, senior director of digital production operations for Fox Sports. “By leveraging AWS technology, we can automatically reformat live action into optimized vertical highlights in near real time, dramatically accelerating turnaround. This allows us to scale premium, editorially sound content across Instagram, TikTok, YouTube Shorts, as well as our owned and operated platforms like the Fox Sports App and Fox One, without sacrificing quality. Going forward, this technology enables an always-on, faster, and more personalized highlight engine that meets fans wherever they watch.”