TetraGT: Tetrahedral Geometry-Driven Explicit Token Interactions with Graph Transformer for Molecular Representation Learning

Jinjia Feng, Zhewei Wei, Taifeng Wang, Zongyang Qiu

Keywords: Molecular Representation Learning, Graph Transformer, Molecular Geometry Pretraining

TL;DR: The Tetrahedral Graph Transformer (TetraGT) directly models molecular geometric tokens with effective interactions through tetrahedral inequalities, enabling enhanced molecular representations and superior accuracy in molecular property prediction.

Abstract:

Molecular representations that fully capture geometric parameters such as bond angles and torsion angles are crucial for accurately predicting important molecular properties including enzyme catalytic activity, drug bioactivity, and molecular spectral characteristics, as demonstrated by extensive studies.
However, current molecular graph representation learning approaches represent molecular geometric parameters only indirectly through combinations of atoms and bonds, neglecting the spatial relationships and interactions between these higher-order geometric structures.
In this paper, we propose \textbf{TetraGT} (\textbf{Tetra}hedral \textbf{G}eometry-Driven Explicit \textbf{T}oken Interactions with Graph Transformer), a novel architecture that directly models molecular geometric parameters.
Based on the spatial solid geometry theory of face angle and dihedral angle inequality, TetraGT explicitly represents bond angles and torsion angles as structured tokens for the first time, directly reflecting their intrinsic role in determining the molecular conformational stability and properties.
Through our designed spatial tetrahedral attention mechanism, TetraGT achieves highly selective direct communication between structural tokens.
Experimental results demonstrate that TetraGT achieves superior performance on the PCQM4Mv2 and OC20 IS2RE benchmarks.
We also apply our pre-trained TetraGT model to downstream tasks including QM9, PDBBind, Peptides and LIT-PCBA, demonstrating that TetraGT delivers excellent results in transfer learning scenarios and shows scalability with increasing molecular size.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Submission Number: 13464