Retrieval-Augmented Dialogue Knowledge Aggregation for expressive conversational speech synthesis

Rui Liu, Zhenqi Jia, Feilong Bao, Haizhou Li

Abstract: Highlights•We propose a novel Retrieval-Augmented Dialogue Knowledge Aggregation CSS model, termed RADKA-CSS.•We introduce a dialogue semantic-style database and design a multi-attribute retrieval module to facilitate style- and semantics-based dialogue retrieval.•We propose a multi-granularity heterogeneous graph modelling mechanism that aims to capture the structural and temporal relations of nodes at different granularities, effectively encoding dialogue semantics and style information.•The experimental results on the DailyTalk benchmarking dataset validate the effectiveness of RADKA-CSS.