AraLive: Automatic Reward Adaption for Learning-based Live Video Streaming

Huanhuan Zhang, Zhuo Liu, Haotian Li, Anfu Zhou, Chuanming Wang, Huadong Ma

Abstract: Optimizing user Quality of Experience (QoE) for live video streaming remains a long-standing challenge. The Bitrate Control Algorithm (BCA) plays a crucial role in shaping user QoE. Recent advancements have seen RL-based algorithms overtake traditional rule-based methods, promising enhanced QoE optimization. Nevertheless, our comprehensive study reveals a pressing issue: current RL-based BCAs are limited to the fixed and formulaic reward functions, rendering them ill-equipped to adapt to dynamic network environments and varied viewer preferences. In this work, we present AraLive, an automatically adaptive reward learning method designed for seamless integration with any existing learning-based approach in live streaming contexts. To accomplish this goal, we construct a dedicated user QoE assessment dataset for live streaming and customize-design an adversarial model that skillfully aligns human feedback with actual network scenarios. We have deployed AraLive in not only the live streaming but also the classic VoD systems, in comparison to a series of state-of-the-art BCAs. The experimental results demonstrate that AraLive not only elevates overall QoE but also exhibits remarkable adaptability to varied user preferences.

Primary Subject Area: [Systems] Transport and Delivery

Relevance To Conference: QoE optimization for live video streaming has been a persistent challenge in multimedia community. The main limitation of current Reinforcement Learning (RL)-based Bitrate Control Algorithms (BCAs) lies in their reliance on fixed and formulaic reward functions, which hampers their ability to adapt to changing network conditions and diverse viewer preferences. In this paper, we introduce an innovative automatically adaptive reward learning approach that aims to not only enhance overall QoE but also demonstrate significant adaptability to various user preferences. We are confident that the designed method will make a meaningful contribution to the multimedia community.

Submission Number: 4322