\section{Conclusion}
We have used field-theoretical tools from statistical physics to derive a nonparametric free energy, which allowed us to produce analytical insights into the pathologies of deep heteroskedastic regression. These insights generalize across models and datasets and provide a theoretical explanation for the need for carefully tuned regularization in these models, due to the presence of sharp phase transitions between pathological solutions.

We have also presented a numerical approximation to this theory, which empirically agrees with neural network solutions to synthetic and real-world data.
Insights from the theory have informed a method to tune the regularization to arrive at well-calibrated models more efficiently than would na\"ively be the case.
Finally, we hope that this work will open an avenue of research for using ideas from theoretical physics to study the collective effects and nonlinear phenomena frequently encountered in large-scale deep learning models \citep{bamler_improving_2018}.


\paragraph{Limitations}
Our FT and subsequent analysis are restricted to regression problems. 
From an uncertainty quantification perspective, the models we discuss only account for the aleatoric uncertainty.
Though our use of regularizers has a Bayesian interpretation, we are not performing Bayesian inference and do not account for epistemic uncertainty \citep{papamarkou_position_2024}. Solving the FT under a fully Bayesian framework would result in stochastic PDE solutions. We leave analysis of this setting to future work.
Additionally, our suggestion to search $\rho = 1-\gamma$ to find good hyperparameter settings appears to be valid, but requires fitting many models.
Ideally, one might hope to use the field theory directly to find optimal regularization settings for real-world models, but our numerical approach is currently not accurate enough for this use case.

\paragraph{Acknowledgements}
Eliot Wong-Toi acknowledges support from the Hasso Plattner Research School at UC Irvine. Alex Boyd acknowledges support from the National Science Foundation Graduate Research Fellowship grant DGE-1839285. Vincent Fortuin was supported by a Branco Weiss Fellowship. Stephan Mandt acknowledges support by the IARPA WRIVA program, the National Science Foundation (NSF) under the NSF CAREER Award 2047418; NSF Grants 2003237 and 2007719, the Department of Energy, Office of Science under grant DE-SC0022331, as well as gifts from Intel, Disney, and Qualcomm.

