How to visualize training dynamics in neural networks

Michael Y. Hu, Shreyans Jain, Sangam Chaulagain, Naomi Saphra

Blogpost Url: https://d2jud02ci9yv69.cloudfront.net/2025-04-28-visualizing-training-87/blog/visualizing-training/

Abstract: Deep learning practitioners typically rely on training and validation loss curves to understand neural network training dynamics. This blog post demonstrates how classical data analysis tools like PCA and hidden Markov models can reveal how neural networks learn different data subsets and identify distinct training phases. We show that traditional statistical methods remain valuable for understanding the training dynamics of modern deep learning systems.

Conflict Of Interest: MYH and NS also wrote "Latent State Models of Training Dynamics."

Submission Number: 48