Stabilizing Continuous-Time Kolen–Pollack Learning with a Scale-Balance Condition

Marc Gong Bacvanski, Liu Ziyin, Tomaso Poggio

Keywords: continuous-time learning dynamics, Kolen–Pollack, biologically plausible learning, heterosynaptic plasticity, feedback alignment

TL;DR: Heterosynaptic plasticity theory reveals a local scale-balance condition that every KP-like implementation has been quietly satisfying through normalization or weight decay; we show that strictly neuron-local bounded activations satisfy it instead.

Abstract: Kolen–Pollack (KP) is a candidate biologically plausible learning rule whose continuous-time formulation eliminates phase separation but suffers from instability in deep networks. Existing discrete-time KP implementations rely on engineered stabilizers (normalization, weight decay, gradient clipping, and adaptive optimizers) that have no obvious counterpart in a continuous-time biological or analog substrate. We analyze continuous-time KP through heterosynaptic plasticity (HSP) theory and identify a local scale-balance condition between the plasticity drive and the decay term that KP's alignment mechanism does not by itself enforce. We show that bounded activations restore this balance, and matches or exceeds layer-local normalization performance on 5 and 10 layer MLPs.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 93