Neighborhood-Adaptive Context Enhancement Learning For Scene Graph Generation

Bowen Zhao, Licheng Zhang, Lei Zhang, Zhendong Mao

Abstract: Conventional scene graph generation methods primarily attempt to implicitly memorize data knowledge within model parameters, yet struggle to excel across all categories, particularly under the influence of long-tailed distributions. Complementing this parameter-based approach, we suggest gathering triplets from similar neighbor instances as extra knowledge. Based on this, we propose a novel Neighborhood-adaptive Context Enhancement Learning (NACEL) network to dynamically select helpful knowledge and integrate it with contextual features for enhanced adaptability. Our method exposes the model to more instances beyond the input, boosting the efficiency and performance of relation prediction. By applying our method on various baselines, extensive experiments on VG dataset have shown that category-level metric mRecall has been significantly improved while instance-level metric Recall has not excessively degraded, which demonstrates our plug-and-play method effectively alleviates biased problem and has the best comprehensive performance.