Fusion of GNN and GBDT Models for Graph and Node Classification

Muhammad Farhan, Nadeem Iqbal Kajla, Malik Daler Ali Awan, Mickaël Coustaty, Muhammad Muzzamil Luqman, Souhail Bakkali

Abstract: The discipline of graph-based machine learning, which focuses on learning from structured graph data, is expanding rapidly. Numerous applications in recommendation systems, bioinformatics, and social network analysis fall within this domain. However, traditional Graph Neural Networks (GNNs) face difficulties when dealing with datasets that frequently contain structured and graph data. Our approach addresses this challenge by creating a proposed fusion model of GNN and Gradient Boosting Decision Trees (GBDTs). We investigated the effectiveness of using the GNN and GBDT based fusion model using logistic regression by combining the embeddings of GNN and GBDT, stacking the predictions from GBDT variants for node classification and graph classification. The generality of the model is tested and validated on one heterogeneous and two homogeneous state-of-the-art datasets with average accuracy of A: Freebase: 0.6875, B: Letters (Low: 96.62, Med: 84.81, High: 78.06), C:Fingerprints:80.45, D:OGBG-MolHIV:89.10 outperforming individual methods. The results indicated that the fusion approach was effective in accurately classifying complete graphs and nodes, although their performance varied depending on the dataset and the characteristics of the graph being analyzed. This shows that the applied technique can get the appropriate results. The supplementary material of our work is publicly available at (https://github.com/mr49online/fusion_model).