Biomedical Abstract Sentence Classification by BERT-Based Reading Comprehension

Cheng-Yang Jiang, Yao-Chung Fan

Abstract: In this paper, we report the investigation of leveraging a reading comprehension model based on BERT for sentence classification in scientific abstracts. Our idea is to reformulate the sentence classification problem as a reading comprehension problem and employ pretrained BERT model to tackle the problem. By the reformulation, we yield state-of-the-art results for scientific abstract classification on two benchmark datasets, i.e., PubMed RCT and NICTA-PIBOSO; our models outperform the best performing models by 4–5% in terms of $$F_{1}$$ F 1 scores. Moreover, we conduct ablations to reveal that the model benefits from the altered training goal of reading comprehension and show that robustness can be achieved by altering sentence order during training. Our code and datasets are available at github ( https://github.com/UDICatNCHU/Scientific-Literature-Sentence-Classification-by-BERT-based-Reading-Comprehension ).