End-To-End Data Characteristics Monitoring for MLOps

Natnaree Jubju, Natawut Nupairoj

Abstract: Machine learning models employ data for gathering insights, making decisions, and generating predictions. As inferenced data fed into the model may drift or shift over time, it may lead to model's performance degradation. Consequently, a model would require re-training. However, model evaluation and frequent re-training might be costly as ground truth labeling can be expensive. Therefore, monitoring data characteristics before and after model deployment can help choosing the appropriate time to re-train the model. This paper proposes a framework of end-to-end data characteristics monitoring within MLOps to provide a solution for smart retraining using a variety of tools for ease of use and cost- effectiveness.