Dead Feature Counts in Sparse Autoencoders Predict Underlying Deep Q Networks' Effectiveness

Coleman DuPlessie

Track: Proceedings Track

Keywords: Sparse Autoencoders, Mechanistic interpretability, Deep Q Networks, Atari, Dead features

Abstract: Sparse autoencoders (SAEs) are machine learning models that can be used to express the inner workings of certain other models as human-interpretable features. While sparse autoencoders work well when applied to language models, there has been little research that investigates the extent to which they generalize to other applications of machine learning. This work investigates the application of SAEs to a deep Q network trained to complete a simple task. We find that, although SAEs tend to perform well and find a number of human-interpretable features, they contain a large number of "dead features" that never activate, which suggests that more research is necessary to adapt SAEs to the unique tasks reinforcement learning models solve. In particular, we note that the most effective deep Q networks trained to complete a task tend to result in sparse autoencoders with a consistent quantity of dead features. This suggests that these sparse autoencoders may in some sense be capturing the "optimal" or "true" number of features needed to solve the toy problem we study, and the high number of dead features may simply imply that additional live features past a certain quantity are unhelpful.

Submission Number: 11