Bayesian Joint Nonlinear System Model Learning, Sensing and Signal Detection in ISAC With Hardware Imperfections

Dawei Gao, Qinghua Guo, Ming Jin, Zhengdao Yuan, Guisheng Liao, Wanqing Li, Yuntao Wu

Published: 2026, Last Modified: 13 Mar 2026IEEE Trans. Wirel. Commun. 2026EveryoneRevisionsBibTeXCC BY-SA 4.0
Abstract: This work addresses the challenges of communication signal detection and direction of arrival (DOA) estimation in integrated sensing and communications (ISAC) systems with hardware imperfections. Conventional signal processing techniques often fail to effectively manage the complex nonlinearities caused by hardware imperfections, such as those introduced by power amplifiers and local oscillators. Recently, deep neural networks (DNNs) have been employed to mitigate the hardware imperfections, which however require a substantial amount of pilot signals for training, leading to unacceptable overhead and impracticality in fast time-varying channels. In this work, we employ an NN to characterize the nonlinear system, and propose a novel iterative approach to joint NN-based nonlinear system model learning, signal detection and DOA estimation. Instead of relying on pilot signals for NN learning, the proposed approach utilizes communication data signals as virtual training samples, enabling more accurate nonlinear model learning, which subsequently enhances signal detection and DOA estimation. A Bayesian framework is applied to the joint problem, wherein the NN parameters, the communication signals and the DOAs are jointly obtained by developing a message passing based inference algorithm. In particular, we impose sparse priors on the weights of the NN, so that overfitting can be better handled, resulting in significant improvement in system modeling performance. Extensive simulation results show that, compared to the state-of-the-art approaches, the proposed one delivers significantly better performance.
Loading