Gaussian Processes for regression in channel equalization

Sebastian Caro, Fernando Pérez-Cruz, Juan José Murillo-Fuentes

Abstract: Linear equalizers underperform in dispersive channels with additive white noise, because optimal decision functions are nonlinear. In this paper we present Gaussian Processes (GPs) for regression as new nonlinear equalizer for digital communication systems. GPs can be cast as nonlinear MMSE, a common criterion in digital communications. Unlike other nonlinear kernel based methods, such as kernel adaline or support vector machines, the solutions produced by GPs are analytical, and the hyperparameters can be readily learnt by maximum likelihood. Hence, we avoid cross-validation or noise estimation, and improve convergence speed. We present experimental results, over linear and nonlinear channel models, to show that GP-equalizers outperform linear and nonlinear state-of-the-art solutions.