Deploying Data Selection Techniques on Dynamic Datasets
Keywords: data selection, subset selection, dynamic datasets, machine learning systems
TL;DR: We present a system for deploying data selection techniques efficiently in practice.
Abstract: Machine learning training data is often dynamic in real-world use cases, i.e., data is added or removed and may experience distribution shifts over time. Models must incorporate this evolving training data to improve generalization, adapt to potential distribution shifts, and adhere to privacy regulations. However, the cost of model (re)training is proportional to how often the model trains and on how much data it trains on. While ML research explores these topics in isolation, there is no end-to-end open-source platform to facilitate the exploration of model retraining and data selection policies and the deployment these algorithms efficiently at scale.
We present Modyn, a platform for model training on dynamic datasets that enables exploring sample-level data selection in practice. Modyn orchestrates continuous training pipelines while optimizing the underlying system infrastructure to support fast access to arbitrary data samples for efficient data selection. Modyn's extensible architecture allows users to run training pipelines without modifying the platform code, and enables researchers to effortlessly extend the system. Modyn is able to reach 80 to 100 % of the throughput compared to loading big chunks of data locally without sample-level data selection.
Primary Subject Area: Active learning, Data cleaning, acquisition for ML
Paper Type: Extended abstracts: up to 2 pages
DMLR For Good Track: Participate in DMLR for Good Track
Participation Mode: In-person
Confirmation: I have read and agree with the workshop's policy on behalf of myself and my co-authors.
Submission Number: 10
Loading