Learning Across the Noise Spectrum: An Approach to Reinforcement Learning with Asynchronous Signals

ICLR 2026 Conference Submission14234 Authors

Keywords: Reinforcement Learning, Robotics, Regularization, SAC, RNNs, Partial Observability

TL;DR: We study reinforcement learning in environments with asynchronous signals, proposing both a theoretical framework to tackle the topic and a novel learning algorithm that improves baselines alternative on asynchronous environments.

Abstract: Reinforcement learning (RL) frameworks assume agents receive complete observation vectors at each timestep. However, real-world robotic systems typically operate in environments with asynchronous signals, i.e. sensors that update at different frequencies. We model \textbf{asynchronous environments} as an instance of a noise-parameterized family of partially observable Markov decision processes (POMDPs). Our primary contribution, **Learning Across the Noise Spectrum (LANS)**, is a novel strategy that exposes the agent to multiple simulated noise regimes during training, implemented using Soft Actor-Critic (SAC) with recurrent neural networks (RNNs). By sampling different asynchronicity rates, we encourage the development of robust estimators. We prove that LANS acts a _time-aware_ regularization term, equivalent to a Jacobian penalty along time-sensitive directions. Experiments on MuJoCo environments with simulated asynchronicity demonstrate that LANS outperforms alternative methods on a variety of tasks—up to a factor of $>1.5\times$ in some instances—offering a solution for robotic systems that must operate with imperfect sensory information.

Primary Area: reinforcement learning

Submission Number: 14234