Alleviating Training Bias with Less Cost via Multi-expert De-biasing Method in Scene Graph Generation

Xuezhi Tong, Rui Wang, Lihua Jing

Abstract: Scene graph generation (SGG) methods have suffered from a severe training bias towards frequent (head) predicate classes. Recent works owe it to the long-tailed distribution of predicates and alleviate the long-tailed problem to conduct de-biasing. However, the "unbiased'' models are in turn biased to tail predicate classes, resulting in a significant performance loss on head predicate classes. The main cause of such a trade-off between head and tail predicates is the fact that multiple predicates from the head or tail ones can be labeled as the ground-truth. To this end, we propose a multi-expert de-biasing method (MED) for SGG that can produce unbiased scene graphs with minor influence on recognizing head predicates. We avoid the dilemma of balancing between head and tail predicates by adaptively classifying the predicates with multiple complementary models. Experiments on the Visual Genome dataset show that MED provides significant gains on mRecall@K without harming the performance on Recall@K, and achieves a state-of-the-art on the mean of Recall@K and mRecall@K.