Embedding and Beamforming: All-Neural Causal Beamformer for Multichannel Speech Enhancement

Andong Li, Wenzhe Liu, Chengshi Zheng, Xiaodong Li

Abstract: Standing upon the intersection of traditional beamformers and deep neural networks, we propose a causal neural beamformer paradigm called Embedding and Beamforming, and two core modules are devised accordingly, namely EM and BM. For EM, instead of estimating spatial covariance matrix explicitly, the 3-D embedding tensor is learned with the network, where the spatial-spectral discriminative information can be implicitly represented. For BM, a network is directly leveraged to derive the beamforming weights so as to implement filter-and-sum operation. To further improve the speech quality, a post-processing module is introduced to further suppress the residual noise. Based on the DNS-Challenge dataset, we conduct the experiments for multichannel speech enhancement and the results show that the proposed system outperforms previous advanced baselines by a large margin in terms of multiple evaluation metrics.