Building Privacy-Preserving Medical Text Models With a Pretrained Transformer

Muhammad Shafiq, Lijing Ren, Gautam Srivastava, Denghui Zhang, Sami Bourouis, Thippa Reddy Gadekallu

Abstract: The rapid advancement of big data and artificial intelligence (AI) in healthcare heightens the urgency for accurate medical text sentiment analysis. The privacy protection of medical data has been a crucial concern due to its sensitivity. The Internet of Medical Things (IoMT) facilitates large-scale data collection at lower cost, enabling precision medicine. However, decentralized IoMT poses novel challenges to centralized standard encryption schemes. In this article, we propose a novel approach to building privacy-preserving sentiment models with a generative pretrained transformer (GPT). We first convert sensitive medical text data into noise-like and distributed one-hot images. Then, we introduce visual cryptography (VC) for lightweight and secure transmission of medical text across public networks in resource-limited IoMT devices. We adopt a cross-domain sentiment analysis framework that finetunes transformer-based language models for accurate sentiment analysis instead of training GPT in sentiment analysis from scratch. Experimental results show that the proposed approach improves the accuracy and effectiveness of sentiment analysis while maintaining privacy, thereby addressing a significant gap in biomedical text analysis.