SHINE: Shielding Backdoors in Deep Reinforcement Learning

Wenbo Guo, Zhuowen Yuan, Jinyuan Jia, Bo Li, Dawn Song

Primary Area: societal considerations including fairness, safety, privacy

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: deep reinforcement learning, trojan backdoor, explanation

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: Recent studies have discovered that similar to supervised classifiers, a deep reinforcement learning (DRL) policy is also vulnerable to backdoor attacks. Existing defenses against backdoor attacks either do not consider RL's unique mechanism or make unrealistic assumptions, resulting in limited defense efficacy, practicability, and generalizability. In this work, we propose SHINE, a novel backdoor shielding method for DRL. SHINE first leverages policy explanation techniques to identify the backdoor triggers and then designs a policy retraining algorithm to eliminate the negative impact of the triggers on backdoored agents. We theoretically prove that SHINE guarantees to improve a backdoored agent's performance in a poisoned environment while ensuring its performance difference in the clean environment before and after shielding is bounded. We further conduct extensive experiments that evaluate SHINE against three mainstream DRL backdoor attacks in various benchmark RL environments. Our results show that SHINE significantly outperforms existing defenses in mitigating these backdoor attacks.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8582