RNA-Scope: Benchmarking RNA Language Models for RNA Sequence Understanding

Hui Wang, Wenjun Lin, Hongwang Xiao, Qiwei Ye, Yaqing Zhang

Track: Track 1: Original Research/Position/Education/Attention Track

Keywords: RNA Language models, RNA Sequence Understanding, Benchmarks, Structure Tasks, Interaction Tasks, Function Tasks

TL;DR: We present RNA-Scope, a comprehensive benchmark of 1,253 experiments that systematically evaluates RNA language models across structure, interaction, and function tasks, addressing key limitations of prior work in capturing RNA biology's complexity.

Abstract: Pre-trained language models (pLMs) have advanced our understanding of RNA biology. However, current evaluation frameworks remain limited in capturing the inherent complexity of RNA, leading to insufficient and biased assessments that hinder their practical applications. Here, we introduce RNA-Scope, a comprehensive benchmarking framework designed to gauge RNA pLMs via structure prediction, interaction classification, and function characterization. This framework includes 1,253 experiments spanning diverse subtasks of varying complexity and enables systematic model comparison with consistent architectural modules. Model assessment shows that generalization of sequence flexibility across RNA families, target contexts, and environmental features remains challenging for existing models. RNA-Scope provides a systematic, robust, and fair evaluation framework to accelerate RNA modeling.

Submission Number: 346