RGP: A Cross-Attention based Graph Transformer for Relational Deep Learning

Divyansha Lachi, Mahmoud Mohammadi, Joe Meyer, Vinam Arora, Tom Palczewski, Eva L Dyer

Keywords: relational deep learning, relational graph transformers, graph transformers, relational databases

Abstract:

In domains such as healthcare, finance, and e-commerce, the temporal dynamics of relational data emerge from complex interactions—such as those between patients and providers or users and products across diverse categories. To be broadly useful, models operating on these data must integrate long-range spatial and temporal dependencies across diverse types of entities, while also supporting multiple predictive tasks. However, existing graph models for relational data primarily focus on spatial structure, treating temporal information merely as a constraint rather than a modeling signal, and are typically designed for single-task prediction.

To address these gaps, we introduce the Relational Graph Perceiver (RGP), a graph transformer architecture for relational deep learning. At its core, RGP employs a Perceiver-style latent bottleneck that integrates signals from different node and edge types into a common latent space, enabling the model to build global context across the entire relational system. It also incorporates a flexible cross-attention decoder that supports joint learning across tasks with disjoint label spaces within a single model. This architecture is complemented by a temporal subgraph sampler, which enhances global context by retrieving nodes beyond the immediate neighborhood. Experiments on RelBench, SALT, and CTU show that RGP delivers state-of-the-art performance, offering a general and scalable solution for relational deep learning with support for diverse predictive tasks.

Submission Type: Full paper proceedings track submission (max 9 main pages).

Submission Number: 108