Improving Optimization in Models With Continuous Symmetry Breaking

Robert Bamler, Stephan Mandt

Abstract: Many loss functions in representation learning are invariant under a continuous symmetry transformation. For example, the loss function of word embeddings (Mikolov et al., 2013) remains unchanged i...