DEPT: Decoupled Embeddings for Pre-training Language Models

Open Webpage

Alex Iacob, Lorenzo Sani, Meghdad Kurmanji, William F. Shen, Xinchi Qiu, Dongqi Cai, Yan Gao, Nicholas Donald Lane

Published: 01 Jan 2025, Last Modified: 18 May 2025ICLR 2025EveryoneRevisionsBibTeXCC BY-SA 4.0