Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining

Open Webpage

Qi Zhang, Tianqi Du, Haotian Huang, Yifei Wang, Yisen Wang

Published: 01 Jan 2024, Last Modified: 16 May 2025ICML 2024EveryoneRevisionsBibTeXCC BY-SA 4.0