A Deep Transfer Learning-Based Object Tracking Algorithm for Hyperspectral Video

Yiming Tang, Yufei Liu, Hong Huang, Chao Zhang, Yuan Li

Abstract: Deep convolutional neural networks (CNNs) have been proved effective in color video visual tracking task. Compared with color video, hyperspectral video contains abundant spectral and material-based information which increases the instance-level discrimination ability. Therefore, hyperspectral video has huge potential for improving the performance of visual tracking task. However, deep trackers based on color video need a large number of samples to train a robust model, while it is difficult to train a hyperspectral video-based CNN model because of the lack of training samples. To tackle with this problem, a novel method is designed on basic of transfer learning technique. At first, a mapping convolutional operation is designed to embed high dimensional hyperspectral video into three channels as color video. Then, the parameters of CNN model learned on color domain are transferred into hyperspectral domain through fine-tuning. Finally, the fine-tuned CNN model is used for hyperspectral video tracking task. The hyperspectral tracker is evaluated on hyperspectral video dataset and it outperforms many state-of-the-art trackers.