Optimizing Depthwise Convolutions on ARMv8 Architecture

Ruochen Hao, Qinglin Wang, Shangfei Yin, Tianyang Zhou, Qingyang Zhang, Songzhu Mei, Siqi Shen, Jie Liu

Abstract: Depthwise convolutions are widely used in lightweight convolutional neural networks (CNNs). The performance of depthwise convolutions is mainly bounded by the memory access rather than the arithmetic operations for classic convolutions so that direct algorithms are often more efficient than indirect ones (matrix multiplication-, Winograd-, and FFT-based convolutions) with additional memory accesses. However, the existing direct implementations of depthwise convolutions on ARMv8 architectures feature a bad trade-off between register-level reuse of different tensors, which usually leads to sub-optimal performance. In this paper, we propose a new direct implementation of depthwise convolutions by means of implicit padding, register tiling, etc. Compared to the existing ones, our new implementations can incur much less communication overhead between registers and cache. Experimental results on two ARMv8 CPUs show that our implementation can averagely deliver 4.88 $$\times $$ performance improvement over the existing direct ones in open-source libraries.