Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration

Qinglin Zhu, Runcong Zhao, Hanqi Yan, Yulan He, Yudong Chen, Lin Gui

Keywords: Large Language Models, Reasoning, Embedding Perturbation, Bayesian Optimisation

TL;DR: We propose an embedding-based search framework that optimises the first token's embedding through controlled perturbation and Bayesian refinement to enhance reasoning accuracy and coherence in large language models with minimal computation.

Abstract: Large Language Models (LLMs) struggle with reasoning due to limited diversity and inefficient search. We propose an embedding-based search framework that optimises the embedding of the first token to guide generation. It combines (1) Embedding perturbation for controlled exploration and (2) Bayesian optimisation to refine embeddings via a verifier-guided objective, balancing exploration and exploitation. This approach improves reasoning accuracy and coherence while avoiding reliance on heuristic search. Experiments demonstrate superior correctness with minimal computation, making it a scalable, model-agnostic solution.

Submission Number: 106