RankDetNet: Delving into Ranking Constraints for Object Detection

Ji Liu, Dong Li

Abstract:

Modern object detection approaches cast detecting ob-
jects as optimizing two subtasks of classification and lo-
calization simultaneously. Existing methods often learn
the classification task by optimizing each proposal sepa-
rately and neglect the relationship among different propos-
als. Such detection paradigm also encounters the mismatch
between classification and localization due to the inherent
discrepancy of their optimization targets. In this work, we
propose a ranking-based optimization algorithm for harmo-
niously learning to rank and localize proposals in lieu of the
classification task. To this end, we comprehensively inves-
tigate three types of ranking constraints, i.e., global rank-
ing, class-specific ranking and IoU-guided ranking losses.
The global ranking loss encourages foreground samples to
rank higher than background. The class-specific ranking
loss ensures that positive samples rank higher than negative
ones for each specific class. The IoU-guided ranking loss
aims to align each pair of confidence scores with the asso-
ciated pair of IoU overlap between two positive samples of
a specific class. Our ranking constraints can sufficiently ex-
plore the relationships between samples from three different
perspectives. They are easy-to-implement, compatible with
mainstream detection frameworks and computation-free for
inference. Experiments demonstrate that our RankDetNet
consistently surpasses prior anchor-based and anchor-free
baselines, e.g., improving RetinaNet baseline by 2.5% AP
on the COCO test-dev set without bells and whistles. We
also apply the proposed ranking constraints for 3D object
detection and achieve improved performance, which further
validates the superiority and generality of our method.