Revisiting zero-shot cross-lingual topic identification: baselines, languages and evaluation

Anonymous

Abstract: In this paper, we revisit cross-lingual topic identification (ID) in zero-shot settings by taking a deeper dive into current datasets, baseline systems and the languages covered. We identify shortcomings in the existing MLDoc evaluation protocol and propose a robust alternative scheme, while also extending the cross-lingual experimental setup to 17 languages. We benchmark several systems that are based on existing multilingual models such as LASER, XLM-R, mUSE, and LaBSE on the new evaluation protocol covering 17 languages. Further, we present a novel Bayesian multilingual document model (MBay) for learning language-independent document embeddings. The model learns to represent the document embeddings in the form of Gaussian distributions, thereby encoding the uncertainty in its covariance. We propagate the learned uncertainties through linear classifiers that benefit in zero-shot cross-lingual topic ID. Our experiments on 17 languages show that the proposed multilingual Bayesian document model performs competitively as compared to other systems based on LASER, XLM-R and mUSE on 8 high resource languages, and outperforms these systems on 9 mid-resource languages. Finally, we consolidate the observations from all our experiments, and discuss points that can potentially benefit the future research works in the area of cross-lingual topic ID.

Paper Type: long