Multiway Multislice PHATE: Visualizing Hidden Dynamics of RNNs through Training

Jiancheng Xie, Lou C Voinov, Noga Mudrik, Gal Mishne, Adam Shabti Charles

Keywords: RNNs, Dimensionality Reduction, Hidden State, Visualization, Hidden Dynamics, Deep Learning

TL;DR: We introduce MM-PHATE, a dimensionality reduction method for visualizing the evolution of RNNs' hidden representation during training.

Abstract: Recurrent neural networks (RNNs) are a widely used tool for sequential data analysis, however, they are still often seen as black boxes of computation. Understanding the functional principles of these networks is key to developing ideal model architectures and optimization strategies. Previous studies often only emphasize the networks' representation post-training, overlooking their evolution process. Here, we present Multiway Multislice PHATE (MM-PHATE), a novel method for visualizing the evolution of RNNs' hidden states. MM-PHATE is a graph-based embedding using structured kernels across the multiple dimensions spanned by RNNs: time, training epoch, and units. We demonstrate on multiple datasets that MM-PHATE uniquely preserves hidden representation community structure among units and identifies information processing and compression phases during training. The embedding allows users to look under the hood of RNNs across training and provides an intuitive and comprehensive strategy to understanding the network's internal dynamics and draw conclusions, e.g., on why and how one model outperforms another or how a specific architecture might impact an RNN's learning ability.

Primary Area: interpretability and explainable AI

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8589