Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examples

Hao Sun, Alihan Hüyük, Daniel Jarrett, Mihaela van der Schaar

Keywords: Accountability, Reinforcement Learning, Batched Control, Accountable Decision-Making, Offline RL, Interpretability in RL

TL;DR: We introduce the Accountable Offline Controller that employs the offline dataset as the Decision Corpus and performs accountable offline control.

Abstract:

Learning controllers with offline data in decision-making systems is an essential area of research due to its potential to reduce the risk of applications in real-world systems. However, in responsibility-sensitive settings such as healthcare, decision accountability is of paramount importance, yet has not been adequately addressed by the literature.
This paper introduces the Accountable Offline Controller (AOC) that employs the offline dataset as the Decision Corpus and performs accountable control based on a tailored selection of examples, referred to as the Corpus Subset. AOC operates effectively in low-data scenarios, can be extended to the strictly offline imitation setting, and displays qualities of both conservation and adaptability.
We assess AOC's performance in both simulated and real-world healthcare scenarios, emphasizing its capability to manage offline control tasks with high levels of performance while maintaining accountability.

Submission Number: 19