Relationship between prediction accuracy and feature importance reliability: An empirical and theoretical study

Jianzhong Chen, Leon Qi Rong Ooi, Trevor Wei Kiat Tan, Shaoshi Zhang, Jingwei Li, Christopher L. Asplund, Simon B. Eickhoff, Danilo Bzdok, Avram J. Holmes, B. T. Thomas Yeo

Abstract: Highlights•Haufe-transformed weights are much more reliable than original regression weights when computing feature importance.•Feature importance reliability is strongly positively correlated with prediction accuracy across phenotypes.•We show mathematically that feature importance reliability is necessary, but not sufficient, for low feature importance error.•We discuss how our theoretical results relate with the reliability of imaging features and behavioral measures.