Thermodynamic Deep Learning: Interpreting Gradient Descent as Energy Flow in a Learning Universe

Gokul Srinath Seetha Ram

Keywords: Deep learning, optimization, thermodynamics, information geometry, entropy, generalization, interpretability

TL;DR: A thermodynamic perspective of deep learning that models gradient descent as an energy–entropy exchange process driving neural networks toward low free-energy equilibrium.

Abstract: We present a thermodynamic interpretation of deep learning, treating gradient descent as an energy–entropy exchange process that evolves neural networks toward equilibrium. Using the Energy–Entropy Framework (EEF), we show that loss minimization corresponds to free-energy reduction, where the learning rate acts as an effective temperature and generalization emerges as a minimal-entropy equilibrium. Experiments on synthetic data with MLP/CNN/Transformer surrogates reveal phase-like transitions and entropy dissipation patterns. This view offers a unified physical perspective on optimization, interpretability, and generalization in deep learning.

Serve As Reviewer: ~Gokul_Srinath_Seetha_Ram1

Submission Number: 43