Leveraging Third-Party LLMs' Annotations for Sensitive Conversational Data Abstractive Summarizaion

Anonymous

Abstract: Previous studies have demonstrated the effectiveness of Large Language Model (LLMs) in various text annotation tasks. However, the use of LLMs as annotators still presents significant limitations that impede their practical efficiency, especially when used through an external API. Particularly, when dealing with sensitive or confidential information in the data to be annotated, relying on a third-party API for LLMs may not be suitable due to privacy concerns. For instance, annotating customer service call transcripts using an LLM for summaries may risk exposing sensitive information discussed during the conversation. In this study, we address this specific challenge by proposing a pipeline that leverages LLM annotations while maintaining the confidentiality of sensitive information submitted through the API.

Paper Type: short

Research Area: Summarization

Contribution Types: NLP engineering experiment, Approaches to low-resource settings, Approaches low compute settings-efficiency

Languages Studied: English, French

Consent To Share Submission Details: On behalf of all authors, we agree to the terms above to share our submission details.