Transferable Lesion-Supervised Speech Representations for Post-Stroke Modelling

Giulia Sanguedolce, Niamh Parkinson, Fatemeh Geranmayeh, Patrick Naylor

Keywords: Embeddings, Clinical Speech, Stroke

TL;DR: Speech-derived lesion representations transfer to unseen structural and cognitive metrics, outperforming clinical and demographic baselines, suggesting a path toward more accessible brain-health assessment where costly neuroimaging is unavailable

Abstract: Characterising post-stroke brain injury typically relies on structural neuroimaging, which is costly, infrastructure-dependent, and poorly suited to repeated or large-scale monitoring; this also creates a data-efficiency problem, because supervised targets in clinical cohorts are often sparsely observed across modalities, limiting the amount of paired speech and clinical data available for downstream modelling. We investigate a transfer learning approach in which representations learned through lesion inference from speech are reused for downstream structural and clinical phenotypes absent from the original training signal. Specifically, patient-level aggregations of out-of-fold predicted lesion probabilities, stacked learner outputs, and uncertainty estimates are derived from a multi-head speech-to-lesion (S2L) model and benchmarked against clinically interpretable speech features, Whisper embeddings, clinical covariates, and demographic baselines under nested cross-validation. S2L representations emerge as the strongest direct predictors of focal lesion bounding-box extent ($R^2=0.241$, $r=0.503$), despite spatial extent being unseen during S2L training, while high-dimensional Whisper encoder embeddings perform best for cognitive outcomes ($R^2=0.486$, $r=0.702$), with further improvement after residual adaptation using S2L representations ($R^2=0.501$). Together, the findings suggest that transferred speech representations can help model post-stroke brain-behaviour relationships when clinical annotations are sparse, supporting more accessible brain-health monitoring.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 50