Beyond Variance Reduction: Understanding the True Impact of Baselines on Policy Optimization

Wesley Chung, Valentin Thomas, Marlos C. Machado, Nicolas Le Roux

Abstract: Bandit and reinforcement learning (RL) problems can often be framed as optimization problems where the goal is to maximize average performance while having access only to stochastic estimates of th...