Object Empowerment-Driven Tool Selection for Exploration in Reinforcement Learning

Faizan Rasheed, Kenzo Clauw, Nicola Catenacci Volpi, Daniel Polani

Keywords: object-empowerment, exploration, intrinsic motivation

TL;DR: use object-tool interactions for empowerment to improve exploration

Abstract:

Tool use enhances problem solving by enabling complex tasks, but remains chal2 lenging for RL agents due to long horizons and sparse, delayed rewards that hinder exploration and learning efficiency. While classic intrinsic motivation (IM) improves exploration, common methods lack focus on object-tool interactions, causing agents to discover many irrelevant details. In this paper, we show how RL
agents can efficiently learn tool use by optimizing object empowerment, an IM measuring control over specific objects. We extend this to multi-tool, multi-object settings, enabling agents to identify key tool-object relations, learn when and how to use tools, and understand their lasting effects. Experiments in MiniHack environments show improved exploration, generalization, and efficiency over PPO. under sparse reward conditions.

Confirmation: I understand that authors of each paper submitted to EWRL may be asked to review 2-3 other submissions to EWRL.

Serve As Reviewer: ~Kenzo_Clauw1

Track: Regular Track: unpublished work

Submission Number: 122