Hardware-Efficient Quantization for Green Custom Foundation Models

Toshiaki Koike-Akino, Chang Meng, Volkan Cevher, Giovanni De Micheli

Keywords: Green AI, Quantization, Foundation Models, Custom Design, Low Power Hardware

TL;DR: Hardware-aware quantization of large foundation models to achieve low-power full-custom design, showing 20-fold reduction in power consumption.

Abstract: We propose a new hardware-efficient quantization (HEQ) for low-power full-custom foundation models. The HEQ jointly optimizes multiplier hardware and weight quantization to minimize the total power consumption. Exploiting power profile of custom multipliers, our method achieves a significant power reduction up to 20 folds.

Submission Number: 17