Improving the Efficiency-Accuracy Trade-off of DETR-Style Models in Practice

Yumin Suh, Dongwan Kim, Abhishek Aich, Samuel Schulter, Jong-Chyi Su, Bohyung Han, Manmohan Chandraker

Abstract: We aim to provide a comprehensive view of the inference efficiency of DETR-style detection models. We explore the effect of basic efficiency techniques and identify the factors that are easy to implement, yet effectively improve the efficiency-accuracy trade-off. Specifically, we investigate the effect of input resolution, multi-scale feature enhancement, and backbone pre-training. Our experiments support that 1) adjusting the input resolution is a simple yet effective way to achieve a better efficiency-accuracy trade-off. 2) Multi-scale feature enhancement can be lightened with a marginal decrease in accuracy, and 3) improved backbone pre-training can further improve the trade-off.