On Interpretability of Linear Autoencoders

Martin Spisák, Radek Bartyzal, Antonín Hoskovec, Ladislav Peska

Abstract: We derive a novel graph-based interpretation of linear autoencoder models easer, slim, and their approximate variants. Contrary to popular belief, we reveal that the weights of these models should not be interpreted as dichotomic item similarity but merely as its magnitude. Consequently, we propose a simple modification that considerably improves retrieval ability in sparse domains and yields interpretable inference with negative inputs, as demonstrated by both offline and online experiments. Experiment codes and extended results are available at https://osf.io/bjmuv/.