Dynamics-Augmented Decision Transformer for Offline Dynamics Generalization

Changyeon Kim, Junsu Kim, Younggyo Seo, Kimin Lee, Honglak Lee, Jinwoo Shin

Keywords: Deep Reinforcement Learning, Dynamics Generalization, Offline Reinforcement Learning, Transformers

TL;DR: We introduce Dynamics Augmented Deicision Transformer for offline dynamics generalization.

Abstract: Recent progress in offline reinforcement learning (RL) has shown that it is often possible to train strong agents without potentially unsafe or impractical online interaction. However, in real-world settings, agents may encounter unseen environments with different dynamics, and generalization ability is required. This work presents Dynamics-Augmented Decision Transformer (DADT), a simple yet efficient method to train generalizable agents from offline datasets; on top of return-conditioned policy using the transformer architecture, we improve generalization capabilities by using representation learning based on next state prediction. Our experimental results demonstrate that DADT outperforms prior state-of-the-art methods for offline dynamics generalization. Intriguingly, DADT without fine-tuning even outperforms fine-tuned baselines.