Enabling Agents to Communicate Entirely in Latent Space

ICLR 2026 Conference Submission5390 Authors

Keywords: Multi-agent systems, Inter-agent latent communication, Latent space reasoning, Compression of latent representations

TL;DR: We send an LLM's last hidden states between agents instead of natural language, and train to compress them preserving utility. This improves success over fine-tuned NL baselines; latents compress to 8 steps preserve performance with big latency cuts.

Abstract: While natural language is the de facto communication medium for LLM-based agents, it presents a fundamental constraint. The process of downsampling rich, internal latent states into discrete tokens inherently limits the depth and nuance of information that can be transmitted, thereby hindering collaborative problem-solving. Inspired by human ''mind-reading'', we propose Interlat (Inter-agent Latent Space Communication), a paradigm that leverages the last hidden states of an LLM as a representation of its mind for direct communication (termed ''latent communication''). An additional compression process further compresses latent communication via entirely latent space reasoning. Experiments demonstrate that Interlat outperforms both fine-tuned chain-of-thought (CoT) prompting and single-agent baselines, promoting more exploratory behavior and enabling genuine utilization of latent information. Further compression not only substantially accelerates inference but also maintains competitive performance through an efficient information-preserving mechanism. We position this work as a feasibility study of entirely latent space inter-agent communication, and our results highlight its potential, offering valuable insights for future research.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 5390