HeteGen: Efficient Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices

Download PDF
Open Webpage

Xuanlei Zhao, Bin Jia, Haotian Zhou, Ziming Liu, Shenggan Cheng, Yang You

Published: 01 Jan 2024, Last Modified: 13 Nov 2024MLSys 2024EveryoneRevisionsBibTeXCC BY-SA 4.0