Select First, Transfer Later: Choosing a Proper Dataset for SRL and GNN Based Transfer Learning

Thais Luca, Aline Paes, Gerson Zaverucha

Abstract: Traditional machine learning algorithms typically operate on data represented as independent points in a high-dimensional feature space, overlooking the relational nature inherent in many real-world domains. Approaches that incorporate relational representations, such as Statistical Relational Learning (SRL) and Graph Neural Networks (GNNs), enable learning from complex, structured data by considering dependencies between entities. However, these methods often share a limitation with traditional models: they generally assume that training and testing data are drawn from the same distribution and share the same feature space. When this assumption fails, models must be retrained from scratch on new data. Transfer Learning offers a solution by leveraging knowledge from one or more source tasks to improve learning in a different target task or domain. Prior work in transfer learning for relational and graph-based settings has primarily focused on what and how to transfer knowledge. Yet, the question of from where to transfer remains underexplored, as transferring from any pre-trained model does not always guarantee improved performance. In this work, we address this issue by proposing a method that estimates the suitability of transfer between relational domains, based on Kullback–Leibler (KL) divergence obtained from a Naive Bayes distribution of the target relational data and the distribution of each considered source model (either SRL or GNN). The best source is selected, and we evaluate this approach by assessing the performance of both SRL and GNN state-of-the-art transfer learning algorithms when provided with each considered source domain. Experimental results demonstrate that selecting transfer pairs based on our strategy leads to improved performance, confirming the practical utility of addressing the “from where to transfer” question in relational domains.

External IDs:dblp:journals/ml/LucaPZ26