Lifted Inference for Faster Training (LIFT) in End-to-End Neural-CRF Models

Yatin Nandwani, Ankit Anand, Mausam ., Parag Singla

Abstract: Several works have explored the use of CRFs as a post-processing step at the end of a neural model to explicitly impose structure in the output space. This has resulted in improved performance in many domains with inherent structure over the output variables e.g., labels over pixels in an image. As an extension, joint training of neural-CRF models has also been explored albeit with limited success. This is due to the high cost of CRF inference which becomes a bottleneck in each iteration of back-propagation. On the other side of the spectrum, there has been tremendous progress in the graphical models community on the topic of lifted inference. Lifted inference algorithms exploit symmetry of the underlying model to significantly reduce the computational cost of graphical model inference. In this paper, we set out to explore the following question: Can the advances in lifted inference be leveraged to speed up the joint training of neural-CRF models? Answering in affirmative, our analysis shows that while pure lifted inference does not help in reducing the cost of training (without compromising the quality), a hybrid approach does the job where we gradually refine the level of lifting as the learning proceeds. As the main contribution of our paper, we present LIFT: Lifted Inference for Faster Training, a generic algorithm for faster joint training of neural-CRF models. Experiments in stereo-vision show that our approach can result in up to 50% reduction in training time without any loss in accuracy.