DREAM: Dual-Memory Reasoning for Anticipatory 3D Perception in Autonomous Driving

Sriram Mandalika, Athira Nambiar

Keywords: Dual Memory Reasoning, Content-based Retrieval, Semantic Segmentation

Abstract: Autonomous driving systems must not only interpret the current scene but also anticipate perceptual challenges such as occlusions, sparse observations, and recurring failure patterns. However, most existing LiDAR semantic segmentation methods operate in a frame-centric manner, processing each scan independently and discarding prior contextual information, which limits robustness to dynamic scene changes. We propose **DREAM**, a **D**ual-memory **REA**soning framework for continual LiDAR se**M**antic segmentation that reframes perception as an experience-driven and anticipatory process. DREAM maintains two complementary memory banks in a strictly online setting: a latent memory that stores compact semantic abstractions of previously observed scenes, and an error memory that records representations associated with uncertain predictions. At each time step, relevant memory entries are retrieved via cosine similarity and integrated into the feature space via a lightweight modulation mechanism, enabling the model to reinforce consistent semantic patterns while suppressing recurring failure modes. The backbone remains frozen, and no past scans are replayed, ensuring computational efficiency and bounded memory growth. Extensive experiments on multiple large-scale LiDAR benchmarks demonstrate that DREAM achieves state-of-the-art performance, with consistent improvements on dynamic and small-scale objects, highlighting the effectiveness of persistent and error-aware memory for robust long-horizon perception.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 12