Domain Identification of Scientific Articles Using Transfer Learning and Ensembles

Adeep Hande, Karthik Puranik, Ruba Priyadharshini, Bharathi Raja Chakravarthi

Abstract: This paper describes our transfer learning-based approach for domain identification of scientific articles as a part of the SDPRA-2021 Shared Task. We experiment with transfer learning using pre-trained language models (BERT, RoBERTa, SciBERT), and these are then fine-tuned for this task. The result shows that the ensemble approach performs best as the weights are being taken into consideration. We propose improvements for future work. The codes for the best system are published here: https://github.com/SDPRA-2021/shared-task/tree/main/IIITT .