Budget-Aware Feature Selection for Reinforcement Learning

Ofek Glick, Argaman Mordoch, Guy Azran, Sarah Keren

Keywords: Safe Reinforcement Learning, Feature Selection, Multi-Objective Learning, Reinforcement Learning

TL;DR: We introduce a reinforcement learning framework that dynamically selects cost-effective features using Safe RL and CMDPs, optimizing decision-making under budget constraints.

Abstract: Real-world reinforcement learning (RL) agents often operate in high-dimensional environments where features obtained from sensors or measurements inform decision-making. However, utilizing all available features can result in significant operational costs. This creates a critical need for budget-sensitive feature selection methods that balance task performance with resource constraints. In this work, we formalize the problem of budget-aware feature selection in RL as a Constrained Markov Decision Process (CMDP) and propose an approach to allow an agent to dynamically toggling features during interaction with the environment. To address the dynamic feature selection challenge, we leverage Safe Reinforcement Learning (Safe RL) methodologies, enabling agents to learn policies that respect strict budgetary constraints during both learning and deployment. By incorporating feature selection into the agent’s policy, the agent can learn to choose features independently, while balancing their costs and their relevance or redundancy for the task.

Submission Number: 2