On Sample Selection for Continual Learning: a Video Streaming Case Study
Keywords: Networking, Video Streaming, Machine Learning, Sample Selection, Continual Learning, Tail Performance
TL;DR: We investigate retraining ML-based video streaming. By estimating the sample-space density, we can prioritize rare samples, improving the ML model tail performance. We can also monitor changes in the density to rationalize when retraining may help.
Abstract: Machine learning (ML) is a powerful tool to model the complexity of communication networks.
As networks evolve, we cannot rely on a train-once-and-deploy strategy.
Retraining models regularly—known as continual learning—is necessary.
Networks often generate massive amounts of data, too much to train on regularly. Yet, to date, there is no established methodology to answer the following key questions:
_With which samples to retrain?_ _When should we retrain?_
We address these questions with Memento, a sample selection system for networking ML models.
Memento maintains a training set with the “most useful” samples to maximize sample space coverage.
This approach benefits rare patterns—the notoriously long ``tail'' in networking—without hurting the average.
Moreover, it allows assessing rationally _when_ it may be helpful to retrain, i.e., when the space coverage changes.
We deployed Memento on Puffer, the live-TV streaming project, and achieve a 14% reduction of stall time, 3.5x the improvement of random sample selection, without significantly impacting video quality.
While this paper focuses on Puffer as a case study, Memento's design does not depend on a specific model architecture; it is likely to yield benefits in other ML-based networking applications.
Area: Networking
Type: Solution
Revision: No
Submission Number: 4
Loading