Abstract: Resting-state functional magnetic resonance imaging (rs-fMRI) has been a promising technique for computer-aided diagnosis of neurodevelopmental brain diseases, e.g., autism spectrum disorder (ASD), due to its sensitivity to the progressive variations of brain functional connectivity. To overcome the challenge of overfitting resulting from small sample size, recent studies began to fuse multi-site datasets for improving model generalization. However, these existing methods generally simply combine multiple sites into single dataset, ignoring the heterogeneity (i.e., data distribution discrepancy) among diverse sources. Actually, distribution alignment, alleviating the inter-site distribution shift, is the fundamental step for multi-site data analysis. In this paper, we propose a novel Transport-based Multi-site Joint Distribution Adaptation (TMJDA) framework to reduce multi-site heterogeneity for ASD diagnosis by aligning joint feature and label distribution using optimal transport (OT). Specifically, with the given target domain and multi-source domains, our TMJDA method concurrently performs joint distribution alignment in each pair of source and target domains, upon which, multiple domain-specific classifiers are further aligned by penalizing decision inconsistency among diverse classifiers for reducing inter-site distribution discrepancy. We evaluated our TMJDA model on the public Autism Brain Imaging Data Exchange (ABIDE) database. Experimental results demonstrate the effectiveness of our method in ASD diagnosis based on multi-site rs-fMRI datasets.