Reinforcement Learning for Intensity Control: An Application to Choice-Based Network Revenue Management

Huiling Meng, Ningyuan Chen, Xuefeng Gao

Keywords: reinforcement learning, continuous time, discretization, revenue management

Abstract: Intensity control is a type of continuous-time dynamic optimization problems with important applications in Operations Research. In this study, we adapt the reinforcement learning framework to intensity control using choice-based network revenue management as a case study, which is a classical problem in revenue management that features a large state space, a large action space and a continuous time horizon. We show that the inherent discretization from jump points, a key feature of intensity control, eliminates the need to discretize the time horizon upfront, which was believed to be necessary because most reinforcement learning algorithms are designed for discrete-time problems. This facilitates computation and significantly reduces discretization error. We lay the theoretical foundation for policy evaluation and develop policy-gradient-based actor-critic algorithms for intensity control. A comprehensive numerical study demonstrates the benefit of our approach versus state-of-the-art benchmarks.

Submission Number: 37