An 8.09TOPS/W Neural Engine Leveraging Bit-Sparsified Sign-Magnitude Multiplications and Dual Adder Trees

Hyochan An, Yu Chen, Zichen Fan, Qirui Zhang, Pierre Abillama, Hun-Seok Kim, David T. Blaauw, Dennis Sylvester

Abstract: The computational complexity of neural networks (NNs) continues to increase, spurring the development of high-efficiency neural accelerator engines. Previous neural engines have relied on two's-complement (2C) arithmetic for their central MAC units (Fig. 29.3.1 top, left). However, gate-level simulations show that sign-magnitude (SM) multiplication is significantly more energy efficient; ranging from 35% (with uniformly distributed operands) to 67% (with normally distributed operands <tex>$(\mu={0}, {\sigma=25})$</tex> ). The drawback of sign-magnitude number representation is that SM addition incurs significant overhead in terms of energy consumption and area, requiring upfront comparison of the sign bits and muxing/control to appropriately select between addition and subtraction (Fig. 29.3.1 center, left). This SM addition overhead substantially offsets the gains from SM multiplication in general purpose computing. One recent effort [1] to employ SM representation in neural computation achieved modest energy improvement at the cost of <tex>$2.5\times$</tex> area increase due to full duplication of the MAC units, which would typically be unacceptable for area-/cost-sensitive IoT applications.