Dual-Branch Memory Network for Visual Object Tracking

Jingchao Wang, Huanlong Zhang, Jianwei Zhang, Mengen Miao, Jiapeng Zhang

Abstract: In recent years, memory-based tracking has become the focus of visual object tracking due to its robustness toward objects with arbitrary forms. Although advanced memory networks have produced excellent results, balancing long-term and short-term memory remains a difficult problem in memory building. Therefore, we propose the dual-branch memory network (DualMN), which divided the memory-building task into long-term and short-term memory-building tasks, avoiding conflict between them skillfully. Specifically, the DualMN consists of a long-term memory branch and a short-term memory branch. The former is dedicated to learning the difference between the new target appearance and the surrounding environment. The latter focuses on learning the essential feature of the target to prevent the target drift. In the tracking process, the long-term memory branch and the short-term memory branch complement each other to achieve more accurate target positioning. Experiments results on OTB2015, NFS, UAV123, LaSOT, and TrackingNet benchmarks show that our DualMN achieves comparable performance to the advanced trackers.