Shared dynamic model aligned hypernetworks for contextual reinforcement learning

Jan Benad, Frank Röder, Martin V. Butz, Manfred Eppe

Keywords: Contextual RL, Zero-shot generalization, Hypernetworks

TL;DR: Shared dynamic model aligned hypernetworks achieve superior zero-shot generalization capabilities in contextual reinforcement learning.

Abstract:

We face the challenge of zero-shot generalization in contextual reinforcement learning problems. A distinction is generally made between two cases: either explicit context information is available for the agent, or it is not and has to be inferred from data.
We propose DMA*-SH, an approach that builds on dynamic model aligned context inference. It emergently forms context representations and is never informed explicitly about the actual contextual situation it is in. We first show that normalization and random masking can significantly improve the encoded context representation.
Second, we enhance context utilization using a hypernetwork which predicts context-dependent weights that are shared between dynamic model, policy, and value function estimation neural modules.
Across a diverse set of contextualized environments, we show that our approach achieves superior results, even compared to context-aware baselines.

Confirmation: I understand that authors of each paper submitted to EWRL may be asked to review 2-3 other submissions to EWRL.

Serve As Reviewer: ~Jan_Benad1, ~Frank_Röder1

Track: Regular Track: unpublished work

Submission Number: 163