Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference

Open Webpage

Nadav Timor, Jonathan Mamou, Daniel Korat, Moshe Berchansky, Oren Pereg, Moshe Wasserblat, Tomer Galanti, Michal Gordon-Kiwkowitz, David Harel

Published: 01 Jan 2025, Last Modified: 15 May 2025ICLR 2025EveryoneRevisionsBibTeXCC BY-SA 4.0