Position Paper: Dual-System Language Models via Next-Action Prediction
Keywords: language models, dual-system model, next-action prediction, mathematical reasoning
Abstract: In current Large Language Model (LLM) practices, each token is appended sequentially to the output. In contrast, humans are capable of revising and correcting what we write. Inspired by this gap, in this position paper, we propose a dual-system to simultaneously model the thought process and the output process via the introduction of action tokens. This is achieved by (a) maintaining two sequences of tokens, which include a thought system simulating the human thought process and an output system for storing responses, and (b) introducing removal tokens as action tokens: when a removal token is generated, it is appended only to the thought system, while simultaneously removing certain tokens from the output system. The model uses both systems for next-action prediction. This method allows the retraction of previously generated tokens in the final response and maintains a record of intermediate steps in the thought system. Our framework enables the training of language models to improve the interaction between the thought and output systems, mirroring the way humans refine their thinking for effective written communication. Moreover, it can be implemented with slight modifications to existing LLM architectures and allows for end-to-end training.
Submission Number: 29
Loading