More Than a Toy: Random Matrix Models Predict How Real-World Neural Representations Generalize

Alexander Wei, Wei Hu, Jacob Steinhardt

Abstract: Of theories for why large-scale machine learning models generalize despite being vastly overparameterized, which of their assumptions are needed to capture the qualitative phenomena of generalizati...