When Do You Need Billions of Words of Pretraining Data?

Yian Zhang, Alex Warstadt, Xiaocheng Li, Samuel R. Bowman

Abstract: Yian Zhang, Alex Warstadt, Xiaocheng Li, Samuel R. Bowman. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.