Difference-Based Graph Attention Networks: A Dual Attention Mechanism for Similarity and Dissimilarity in Graph Learning

Rafael Artiñano Muñoz, Lucía Prieto Santamaría, Aurora Pérez-Pérez

Keywords: Graph Neural Networks, dissimilarity, orthogonality

TL;DR: We employ a second, dissimilarity-based attention, combined with orthogonal projections and Iwasawa decompositions, to enhance dot-product attention predictions.

Abstract:

Most Graph Neural Networks (GNNs) rely solely on similarity-based attention
mechanisms, limiting their ability to distinguish nodes that are structurally similar
but semantically distinct. We introduce Difference-Based Graph Attention Net-
work (DGAT), a novel architecture that integrates both similarity and dissimilarity
attention within a unified geometric framework. DGAT models contrastive rela-
tionships using orthogonal projections and wedge-product approximations, cap-
turing richer feature interactions beyond alignment. Our formulation is grounded
in a generalized Iwasawa–Cayley decomposition, where the combination of sim-
ilarity and dissimilarity attention correspond to orthogonal, scaling, and shifting
operations. We also connect its behavior to discrete analogs of differential opera-
tors and function orthogonality, establishing a principled geometric interpretation.
Experiments across homophilic OGB graphs, specially in OGBl-PPA, and het-
erophilic benchmarks show that DGAT consistently outperforms GAT, GATv2,
and Graph Transformer architectures, especially in settings requiring fine-grained
representational contrast or role differentiation.

Primary Area: learning on graphs and other geometries & topologies

Submission Number: 11543