Enhancing Multimodal Survival Prediction with Pathology Reports in Hyperbolic Space
Keywords: Survival Prediction, Computational Pathology, Multimodal Medical Image Analysis
Abstract: Cancer survival prediction using computational pathology has emerged as a crucial tool for diagnosis and treatment planning. Current approaches primarily rely on Whole Slide Images (WSIs) and genomic data, but face significant challenges in capturing the logical relationships between visual features and survival outcomes with limited supervision. While pathology reports could potentially serve as a semantic bridge between WSIs and survival time, existing methods overlook the inherent hierarchical relationships between textual descriptions and visual features, where pathology terms represent more abstract concepts and individual terms may correspond to multiple image regions. To address these challenges, we propose HyperSurv, a novel framework that leverages hyperbolic geometry to model the hierarchical relationships between WSIs and pathology reports. Our key insight is that hyperbolic space naturally captures both the entailment structure between generic report concepts and specific visual features, as well as the one-to-many relationships between pathology terms and image regions. HyperSurv enforces these relationships through hyperbolic cones while identifying survival-relevant features via attention pooling. Extensive experiments on four TCGA cancer datasets demonstrate that our approach achieves state-of-the-art survival prediction performance by effectively modeling these multi-modal hierarchical relationships.
Primary Area: applications to computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 2214
Loading