Double-Bayesian Learning

Stefan Jaeger

Keywords: Artificial Intelligence, Machine Learning, Learning Theory, Bayesian Inference, Neural Networks, Information Theory, Optimization, Stochastic Gradient Descent, Golden Ratio

TL;DR: This paper describes a new double-Bayesian inference framework and its implications for neural network training.

Abstract: Contemporary machine learning methods will try to approach the Bayes error, as it is the lowest possible error any model can achieve. This paper postulates that any decision is composed of not one but two Bayesian decisions and that decision-making is, therefore, a double-Bayesian process. The paper shows how this duality implies intrinsic uncertainty in decisions and how it incorporates explainability. The proposed approach understands that Bayesian learning is tantamount to finding a base for a logarithmic function measuring uncertainty, with solutions being fixed points. Furthermore, following this approach, the golden ratio describes possible solutions satisfying Bayes' theorem. The double-Bayesian framework suggests using a learning rate and momentum weight with values similar to those used in the literature to train neural networks with stochastic gradient descent.

Primary Area: Learning theory

Submission Number: 5459