Overcoming Oscillations in Quantization-Aware Training

Markus Nagel, Marios Fournarakis, Yelysei Bondarenko, Tijmen Blankevoort

Abstract: When training neural networks with simulated quantization, we observe that quantized weights can, rather unexpectedly, oscillate between two grid-points. The importance of this effect and its impac...