RankSearch: An Automatic Rank Search Towards Optimal Tensor Compression for Video LSTM Networks on Edge

Changhai Man, Cheng Chang, Chenchen Ding, Ao Shen, Hongwei Ren, Ziyi Guan, Yuan Cheng, Shaobo Luo, Rumin Zhang, Ngai Wong, Hao Yu

Abstract: Various industrial and domestic applications call for optimized lightweight video LSTM network models on edge. The recent tensor-train method can transform space-time features into tensors, which can be further decomposed into low-rank network models for lightweight video analysis on edge. The rank selection of tensor is however manually performed with no optimization. This paper formulates a rank search algorithm to automatically decide tensor ranks with consideration of the trade-off between network accuracy and complexity. A fast rank search method, called RankSearch, is developed to find optimized low-rank video LSTM network models on edge. Results from experiments show that RankSearch achieves a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$4.84 &gt;$</tex> reduction in model complexity, and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1.96\times$</tex> speed-up in run time while delivering a 3.86% accuracy improvement compared with the manual-ranked models.