Personalized LLM Response Generation with Parameterized User Memory Injection
Abstract: Large Language Models (LLMs) have exhibited remarkable proficiency in comprehending and generating natural language. On the other hand, personalized LLM response generation holds the potential to offer substantial benefits for individuals. However, existing work struggles with efficiently incorporating user information for LLM personalization. In this study, we draw inspirations from real-world bionic memory mechanism to propose a novel parameterized **M**emory-**i**njected approach using parameter-efficient fine-tuning (PEFT) combined with a Bayesian Optimisation searching strategy to achieve **L**LM **P**ersonalization(**MiLP**). Our MiLP takes advantage from the alignment between real-world memory mechanism and the LLM's architecture. Extensive experiments have shown the superiority and effectiveness of MiLP. To encourage further research into this area, we are releasing our implementation code.
Paper Type: Long
Research Area: Dialogue and Interactive Systems
Research Area Keywords: Dialogue and Interactive Systems,Human-Centered NLP,Generation
Contribution Types: Model analysis & interpretability
Languages Studied: English
Submission Number: 3516
Loading