Bibimbap : Pre-trained models ensemble for Domain Generalization

Jinho Kang, Taero Kim, Yewon Kim, Changdae Oh, Jiyoung Jung, Rakwoo Chang, Kyungwoo Song

Abstract: Highlights•We generalized the model via weight ensembling under an extreme distribution shift.•Our framework solves the underfitting problem of ensembles of pre-trained models.•We achieved up to 16.3% AUROC improvement over the baseline on the benchmark dataset.•Our model settles into a stable minima, which we verify on the loss surface.•We analyzed the correlation between model diversity and domain generalization.