Bag of Experts Architectures for Model Reuse in Conversational Language Understanding

Rahul Jha, Alex Marin, Suvamsh Shivaprasad, Imed Zitouni

Abstract: In neurophysiological time series, strong neural oscillations are observed in the mammalian brain, and the natural processing tools are thus centered on narrow-band linear filtering.As this approach is too reductive, we propose new methods to represent these signals.We first focus on the study of phase-amplitude coupling (PAC), which consists in an amplitude modulation of a high frequency band, time-locked with a specific phase of a slow neural oscillation.We propose to use driven autoregressive models (DAR), to capture PAC in a probabilistic model. Giving a proper model to the signal enables model selection by using the likelihood of the model, which constitutes a major improvement in PAC estimation.%We first present different parametrization of DAR models, with fast inference algorithms and stability discussions.Then, we present how to use DAR models for PAC analysis, demonstrating the advantage of the model-based approach on three empirical datasets.Then, we explore different extensions to DAR models, estimating the driving signal from the data, PAC in multivariate signals, or spectro-temporal receptive fields.Finally, we also propose to adapt convolutional sparse coding (CSC) models for neurophysiological time-series, extending them to heavy-tail noise distribution and multivariate decompositions. We develop efficient inference algorithms for each formulation, and show that we obtain rich unsupervised signal representations.