SuMe: A Dataset Towards Summarizing Biomedical Mechanisms

Anonymous

Abstract: Can language models read biomedical texts and explain the biomedical mechanisms discussed? In this work we introduce a biomedical mechanism summarization task. Biomedical studies often investigate the mechanisms behind how one entity (e.g., a protein or a chemical) affects another in a biological context. The abstracts of these publications often include a focused set of sentences that present relevant supporting statements regarding such relationships, associated experimental evidence, and a concluding sentence that summarizes the mechanism underlying the relationship. We leverage this structure and create a summarization task, where the input is a collection of sentences in an abstract and the output includes the main relationships and a natural language sentence that summarizes the mechanism. Using a small amount of manually labeled mechanism sentences, we train a mechanism sentence classifier to filter a large biomedical abstract collection and create a summarization dataset with 22k instances. We also introduce a pretraining conclusion generation task with 611k samples. Our benchmarking experiments with large language models show that the pretraining is helpful for the original task, but the model performance isn't still satisfactory and this task presents significant challenges in biomedical language understanding and summarization.