Deep Reinforcement Learning-based Multi-Target Tracking and Association in Cooperative Millimeter-Wave ISAC Systems

Tongkai LI, Weifeng Zhu, Shuowen Zhang, Jiannong Cao, Shuguang Cui, Liang Liu

Keywords: Integrated sensing and communication (ISAC), multi-target tracking, millimeter wave, deep reinforcement learning

Abstract: This paper investigates the multi-target tracking (MTT) problem in a cooperative millimeter-wave (mmWave) integrated sensing and communication (ISAC) system, where distributed base stations (BSs) emit directional beams to probe the states of associated targets. Owing to the complicated environment and target mobilities, the target-BS associations need to be dynamically adjusted to facilitate stable MTT performance during the tracking interval. In this work, we propose a unified deep reinforcement learning (DRL)-based framework to perform joint target state estimation, target-BS association adjustment, and beam prediction tasks. Specifically, a dynamic graph neural network (DGNN) is first developed to capture the spatio-temporal features among targets and perform target state estimation. In particular, an advantage actor-critic (A2C)-based controller is then proposed for flexible target-BS association adjustment and beam prediction under a novel auto-regressive policy, which ensures that the target‑BS association constraints can be strictly satisfied. Numerical results demonstrate that the proposed scheme can adaptively update associations in response to target mobility, thereby significantly reducing the tracking error.

Submission Number: 12