A Flexible Approach to Deliberation Cost in the Option-Critic Architecture

Augusto Antônio Fontanive Leal, Mateus Begnini Melchiades, Gabriel de Oliveira Ramos

Keywords: Reinforcement Learning, Temporal Abstraction, Options Framework, Flexible Deliberation Cost

Abstract: Temporal abstraction, frequently modeled using the options framework, enables agents to perform temporally extended actions, optimizing intrinsic policies, termination functions, and policies over options without the need for assigning extra rewards. In this context, the deliberation cost emerges as a crucial component, as it penalizes the premature termination of options, promoting more efficient use of computational resources and accelerating the agent's response in dynamic environments. We propose a flexible and adaptable approach to the deliberation cost, dynamically adjusting it based on the termination decisions of the options. Our results indicate that this approach not only improves learning efficiency but also contributes to the specialization and effectiveness of the options, enabling superior performance.

Type Of Paper: Full paper (max page 8)

Anonymous Submission: Anonymized submission.

Submission Number: 28