Habitizing Diffusion Planning for Efficient and Effective Decision Making

Haofei Lu, Yifei Shen, Dongsheng Li, Junliang Xing, Dongqi Han

TL;DR: A general framework for habitizing diffusion planning for efficient decision making

Abstract:

Diffusion models have shown great promise in decision-making, also known as diffusion planning. However, the slow inference speeds limit their potential for broader real-world applications.
Here, we introduce **Habi**, a general framework that transforms powerful but slow diffusion planning models into fast decision-making models, which mimics the cognitive process in the brain that costly goal-directed behavior gradually transitions to efficient habitual behavior with repetitive practice.
Even using a laptop CPU, the habitized model can achieve an average **800+ Hz** decision-making frequency (faster than previous diffusion planners by orders of magnitude) on standard offline reinforcement learning benchmarks D4RL, while maintaining comparable or even higher performance compared to its corresponding diffusion planner. Our work proposes a fresh perspective of leveraging powerful diffusion models for real-world decision-making tasks.
We also provide robust evaluations and analysis, offering insights from both biological and engineering perspectives for efficient and effective decision-making.

Lay Summary: Decision-making models powered by AI have become increasingly powerful — but also increasingly slow. This poses a problem when speed is critical, such as in robotics or motion planning. Inspired by how humans learn — starting with deliberate thinking and gradually forming fast habits — we developed Habi, a method that transforms slow, high-performing decision models into fast ones without sacrificing quality. Even on a regular laptop, Habi can make over 800 decisions per second, far outpacing previous methods, while still achieving equal or better results. This opens the door for using powerful AI planning models in real-time applications.

Link To Code: https://bayesbrain.github.io/

Primary Area: Reinforcement Learning->Batch/Offline

Keywords: Offline Reinforcement Learning, Diffusion Planning, Variational Bayes, Diffusion Models

Submission Number: 1793