FM-EAC: Feature Model-based Enhanced Actor-Critic for Multi-Task Control in Dynamic Environments

Quanxi Zhou, Wencan Mao, Manabu Tsukada, John C.S. Lui, Yusheng Ji

Keywords: Reinforcement Learning, Actor-Critic, Unmanned Aerial Vehicle

TL;DR: We propose a generalized algorithm, FM-EAC, that integrates planning, acting, and learning for multi-task control in dynamic environments.

Abstract: Model-based reinforcement learning (MBRL) and model-free reinforcement learning (MFRL) evolve along distinct paths but converge in the design of Dyna-Q (Sutton and Barto 2018). However, modern RL methods still struggle with effective transferability across tasks and scenarios. Motivated by this limitation, we propose a generalized algorithm, FM-EAC, that integrates planning, acting, and learning for multi-task control in dynamic environments. FM-EAC combines the strengths of MBRL and MFRL and improves generalizability through the use of novel feature-based models and an enhanced actor-critic framework. Simulations in both urban and agricultural applications demonstrate that FM-EAC consistently outperforms many state-of-the-art MBRL and MFRL methods. More importantly, different sub-networks can be customized within FM-EAC according to user-specific requirements.

Primary Area: reinforcement learning

Submission Number: 9290