Distribution Enforcement via Random Probe: Active Distributional Constraints for Robust Deep Learning

Keywords: variational autoencoders, vae

Abstract: Deep learning models rely on distributional assumptions about latent representations, yet these assumptions are rarely explicitly enforced during training. We propose Distribution Enforcement via Random Probe (DERP), a framework that enforces distributional constraints through statistical testing integrated into backpropagation. Our approach explores whether explicit enforcement can improve distributional compliance compared to standard approaches that rely on emergent properties. We evaluate DERP on variational autoencoders using CIFAR-10 and CelebA datasets, showing improved distributional compliance in some cases (KS distance 0.037 vs 0.057 on CelebA) while demonstrating active distributional enforcement during training. DERP maintains computational efficiency with minimal overhead (0-4\%), suggesting potential for broader applications in probabilistic machine learning. The implementation is open source and available for reproduction at https://github.com/belindamo/derp.

Submission Number: 229