When Your Friendly Agent Is More Than Just Friendly: Revealing Side Effects of Using Style Features on LLM

ACL ARR 2025 July Submission1394 Authors

Abstract: Many recent studies use style or persona features—such as “empathetic” or “professional”—to steer agents' behaviors toward desired styles. However, the unintended stylistic side effects introduced by these features remain underexplored. This paper identifies and controls such side effects across commonly used style features, revealing significant cross-feature interference. We conducted a comprehensive survey of recent papers to extract widely used style features and performed empirical analysis using synthetic agent-agent dialogues. Our findings show that many features exhibit strong correlations with others and that their influence often bleeds into unrelated traits. To address this, we design and evaluate counter-strategies that neutralize these effects. Our work demonstrates the existence of such side effects and raises questions about large language models’ faithfulness in following style prompts, offering practical recommendations for safe and targeted style control in LLM-based agents.

Paper Type: Short

Research Area: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Research Area Keywords: evaluation and metrics, applications, dialogue state tracking, conversational modeling;

Contribution Types: NLP engineering experiment

Languages Studied: English

Reassignment Request Area Chair: This is not a resubmission

Reassignment Request Reviewers: This is not a resubmission

A1 Limitations Section: This paper has a limitations section.

A2 Potential Risks: N/A

B Use Or Create Scientific Artifacts: Yes

B1 Cite Creators Of Artifacts: Yes

B1 Elaboration: We built on top of existing datasets to construct our synthetic dataset.

B2 Discuss The License For Artifacts: No

B2 Elaboration: All data comes from public domain that does not have a license.

B3 Artifact Use Consistent With Intended Use: No

B3 Elaboration: All data comes from academic sources that are open for free use.

B4 Data Contains Personally Identifying Info Or Offensive Content: No

B4 Elaboration: No personal data for identification or offensive content.

B5 Documentation Of Artifacts: N/A

B6 Statistics For Data: Yes

B6 Elaboration: We create our synthetic dataset and discuss summary stat for it.

C Computational Experiments: Yes

C1 Model Size And Budget: No

C1 Elaboration: we use api calls only.

C2 Experimental Setup And Hyperparameters: N/A

C2 Elaboration: we do not need to train any models.

C3 Descriptive Statistics: Yes

C3 Elaboration: We have evaluation results, for which we give statistics

C4 Parameters For Packages: No

C4 Elaboration: We only use common packages that do not require special attention.

D Human Subjects Including Annotators: No

D1 Instructions Given To Participants: N/A

D2 Recruitment And Payment: N/A

D3 Data Consent: N/A

D4 Ethics Review Board Approval: N/A

D5 Characteristics Of Annotators: N/A

E Ai Assistants In Research Or Writing: Yes

E1 Information About Use Of Ai Assistants: No

E1 Elaboration: We only use it to help us review our writings, not the research or writing itself.

Author Submission Checklist: yes

Submission Number: 1394