Orthogonalized SGD and Nested Architectures for Anytime Neural Networks

Chengcheng Wan, Henry Hoffmann, Shan Lu, Michael Maire

Abstract: We propose a novel variant of SGD customized for training network architectures that support anytime behavior: such networks produce a series of increasingly accurate outputs over time. Efficient a...