Intermediate Layers Can Be Self-Hard Negative Generator For Large Language Model Based Recommendation

ICLR 2026 Conference Submission18825 Authors

Keywords: Sequential Recommendation; Large Language Model Recommendation;

Abstract:

Large language models(LLMs) have gained significant attention for their usage in recommender systems.
One typical method to adapt LLMs for recommendation is Supervised Fine-tuning(SFT), and subsequent studies introduce preference learning to incorporate negative samples into the training process.
However, the negative samples used in existing preference learning methods are sampled at the sequence-level in an offline process, making them less discriminative and informative when adapting LLMs to recommendation tasks with large negative item spaces.
To address these challenges, we propose ILRec, a novel preference fine-tuning framework for LLM-based recommender systems, which utilizes self-hard negative signals extracted from intermediate layers to enhance preference learning for LLMs. Specifically, we first extract self-hard negative tokens from intermediate layers, which serve as fine-grained negative signals and dynamically reflect the model's preference learning process. To incorporate these negative signals into training, we devise a fine-tuning framework consisting of two components: cross-layer preference optimization and cross-layer preference distillation, which enables the model to effectively distinguish the negative signals and enhance the informativeness of negatives generated by intermediate layers. Additionally, we introduce a small collaborative filtering model to assign reward to each penalized token, preventing potential over-penalization of false negatives. Extensive experiments on three datasets demonstrate ILRec’s effectiveness in enhancing the performance of LLM-based recommender systems.

Primary Area: generative models

Submission Number: 18825