Adeia Blog

All Blogs

May 15, 2024

ACM Mile High Video 2024 Recap: User Experience is the Research Wave of the Future

ACM Mile High Video 2024 Recap: User Experience is the Research Wave of the Future

Adeia recently attended and exhibited at the ACM Mile High Video (MHV) 2024 conference in Denver. This event is a premier forum for discussion about technology research and development in the video industry. Like many other conferences of this nature, it focuses both on the current state of the technology as well as forward-looking technologies. The following is a summary of some of the things we saw and heard that are not only areas of interest for us, but also active areas of research throughout the industry. 

Compression 

You can’t have a conference about video technology without discussing codecs, and MHV 2024 was no exception. The key takeaway in the area of codecs from this year’s conference is that compression has been progressively improving. 

However, there is also broad agreement among the video community that despite the way coding has always worked (such as using block-based encoding on 2D pictures), new paradigms need to come to the forefront. This is because, as the world moves toward the more widespread use of immersive video technologies such as light fields, mesh and point clouds, the traditional block-based encoding methods that have worked so well for 2D video will no longer be sufficient or possibly even applicable. 

Next-generation codecs will have to adapt to the type of content they are encoding, whether 2D video or some other type of 3D/volumetric/immersive video. Even different use cases such as traditional video streaming, live streaming, videoconferencing, cloud gaming, VR and others will all require versatile and adaptable approaches to encoding. 

Networks are becoming more capable, but compression also needs to keep pace so that all types of networks — wired and wireless alike — can handle the delivery of these new types of data, which are expected to create an explosion in data volume in the coming years. 

Compression will also need to be “application aware,” meaning that the codecs will have to be adaptable to the type of content being compressed. For example, room-scale mobile and XR applications will be treated differently than those on more seated/fixed XR applications or even 3D video streaming of a live sporting event. 

Another fascinating consideration for next-gen codecs is that, over time, an increasing amount of video traffic is expected to be viewed primarily by machines, not just humans. For instance, an AI-powered video surveillance system might be monitored by a machine that only alerts the human operator when it “sees” certain characteristics in video served to it. Since computer vision senses video differently than the human eye, video intended for machine viewing can be encoded in such a way to optimize for machine-to-machine applications.  

Low Latency

As streaming has become more widespread, especially for live sports and broadcasts, latency has become an area of focus for the industry. Typically, a live sports broadcast over cable TV carries a latency less than 10 seconds. But latency for streaming over-the-top (OTT) varies somewhat from this and may even be higher than cable. 

With the advent of other technologies like real-time microbetting, improvements in latency become critical. Microbetting is when a wager is placed on the outcome of some highly specific aspect of a sports event. For example, in tennis you might bet on which player is going to win the next point. In hockey or soccer, you might bet on which team will score the next goal. The betting period might only last a few seconds, and the odds change in real time based on a variety of metrics as well as on how many bets are being placed. 

In such scenarios even small delays in the broadcast can become costly. If I’m just watching the game and have no money riding on the outcome, I might be willing to tolerate quite a bit of latency. But if I’m placing bets on the outcome of small events inside a game, I need the stream to be as close to real time as possible, usually measured as one second or less for these kinds of applications. 

This low level of latency is important for other video-based applications as well, such as cloud gaming and videoconferencing, which both need to feel natural in order to be usable. For VR applications, tolerances are even lower: users experience nausea if the latency is higher than about 20 milliseconds.  

QUIC

One exciting approach to reducing latency for a variety of streaming video-based applications is the use of particular transport protocols. While protocols like WebRTC continue to be of interest to the community, QUIC has been gaining attention with the establishment in 2022 of an IETF working group on Media-over-QUIC (MOQ). QUIC can be used in browser (i.e., HTTP/3 web transport) and non-browser (i.e., raw QUIC) environments. The QUIC solution targets various applications including live streaming, cloud gaming, remote desktop, videoconferencing and eSports. 

A promising study presented at the conference investigated the potential benefits of QUIC stream prioritization features, which can improve both latency and resource utilization in the transport layer. QUIC breaks a transmission into multiple streams and assigns different priorities to each stream to optimize the delivery of a transmission with much lower latency. An intelligent application-aware scheme that utilized this property allocated different video frame types (I, P and B) to streams with different priorities, resulting in improved metrics and a better user experience. 

Open Caching

A next-generation streaming architecture, open caching, helps address some of the major bottlenecks inherent in traditional internet infrastructure that are exacerbated by high volumes of streaming media. Instead of serving up streaming content from highly centralized locations, open caching places content caches deep within service provider networks. 

This approach enables network operators to deliver content from locations closer to their customers, improving reliability and latency for users while enhancing operational efficiency for the service provider. In a lively panel discussion, participants agreed that enticing public content delivery networks (CDNs) to participate in a standards-setting discussion in this ecosystem with the internet service providers (ISPs) has been a challenge. 

“CDN Leeching” - A New Form of Piracy

Speaking of CDNs, one fascinating topic that came up at MHV is the concept of CDN leeching, which is an insidious new form of content piracy affecting OTT service providers. Using this method, pirates can procure a token that enables them to access content within the provider’s CDN and serve it to multiple unauthorized, illegitimate users. 

Providers have been able to combat piracy by employing single-use tokens, which guarantee that a given piece of content cannot be replayed. However, the single-use token requires the infrastructure of security components to be scaled appropriately, so this approach has not become a widespread best practice. 

Instead, a novel approach presented at the conference employed methods of rejecting tokens with a probabilistic frequency, meaning that anyone using an illegitimate token will encounter far more rejections than someone who is not reusing a token. This approach ensures, at a probability setting of 1/10000 for example, that a group of 5,000 pirates using the same token would face a near-certain rejection within one minute, while a legitimate user would face only around a 4 percent chance of rejection in a 15-minute window. 

Content Steering

A growing amount of research is focused on improving the user experience. Video service providers today typically operate through multiple CDNs. Traditionally, choices were made about which CDN to use in a given situation based on the service provider’s requirements. The recent focus on prioritizing user experience opens the door to “content steering”, i.e., dynamically switching video streaming sessions between CDNs.  

CDNs themselves are independent, meaning that traditionally, there is no coordination between them. A new standard for content steering, however, can enable multi-CDN communication by allowing a CDN to manage its traffic load via communication with other competing CDNs over the steering server without exposing its internal performance data. As CDNs negotiate with each other based on their resources and load, this approach benefits the user: it helps to coordinate a multi-CDN strategy in order to improve the experience. 

Active Switching

This approach, like content steering, is also focused on the user experience. With active switching, the service provider selects the best CDN endpoint for each video segment. This can be very dynamic, varying between the most cost-effective CDN endpoint, the fastest CDN endpoint or whichever endpoint is most capable of delivering the optimal user experience. By measuring real-time user conditions and using them as a basis for decision-making, this technology remains focused on optimizing the user experience. 

Diversity in Streaming

While attending the conference, our team had the opportunity to talk to a few of our colleagues in the industry, including those who are hard at work to foster diversity at Mile High Video and within the industry as a whole. 

We heard from Tamar Shoham, Chief Technology Officer at Beamr: “Another trend that was obvious this year is the positive impact brought by MHV ongoing support of the Women In Streaming Media group. This was primarily evident in the diversity of the panelists on the excellent panels, as well as the introduction to the Women in Streaming Media Allyship program and the very vibrant Women in Streaming Media meetup.”

Victoria Tuzova, Business Operations at Elecard, shares insights from the speaker engagement team at Women in Streaming Media: “Our efforts at MHV 2024 aimed to elevate the presence of female speakers. Building on last year's focus on diversity and the need for greater integration, we successfully increased the participation of female speakers to 40% per panel this year. Moving forward, we are committed to maintaining this level of representation at future events to further enhance female involvement in the streaming industry.”

Graceful Variation

Our team also had a chance to exhibit and demo our own graceful variation technology. Graceful variation is an innovation we’ve designed to deliver more consistent video quality over longer durations of time, whether you’re watching a live stream or locked into a video game. 

Abrupt changes in streaming quality, for example, jumping from 1080p to 420p, are often jarring and can significantly impact the user experience. They’re caused by sudden changes in connectivity or network quality. But by using dynamic playout through graceful variation, we can reduce abrupt jumps in quality and provide a smoother shift to mitigate perceivable changes for viewers.

 

ACM Mile High Video 2024 Recap: User Experience is the Research Wave of the Future

Exploring the Impact of Artificial Intelligence Applications on the Semiconductor Sector

Technology-enabled Strategies Form Basis for Differentiation for Video Entertainment

Clickable Objects: A Convergence of Video and e-Commerce

DJ Lal

Senior Director Advanced R&D, Media IP at Adeia

Dhananjay (DJ) is responsible for roadmap definition, strategy and R&D activities in Adeia’s Media CTO office. Prior to Adeia, DJ was Senior Director for Emerging Technologies and Platforms at Charter Communication, where he built an R&D team focused on network-powered Gaming, AR/VR, holographic / light field communication and ML/AI applied to Quality-of-Experience delivery on the network. He has held positions across research, product engineering and product management across various organizations like Time Warner Cable, Eaton, Emerson and Bosch. He also served as Board Member and Network Architecture Workgroup Chair at the Immersive Digital Experiences Alliance (IDEA) and has 16 issued U.S. patents. DJ has a BE in Electronics and Communication Engineering from Indian Institute of Technology, a Ph.D. in Computer Science from University of Cincinnati and an MBA in general management from Carnegie Mellon University.