Structure-Informed Deep Reinforcement Learning for Inventory Management

Alvaro Maggiar, Sohrab Andaz, Akhil Bagaria, Carson Eisenach, Dean Foster, Omer Gottesman, Dominique Perrault-Joncas

Keywords: Inventory Management, Reinforcement Learning

Abstract: This paper explores the application of deep reinforcement learning (DRL) to classical inventory management problems while incorporating theoretical insights from traditional operations research. We demonstrate that a simple DRL implementation using DirectBackprop can effectively handle diverse scenarios including multi-period systems with lost sales, lead times, perishability, dual sourcing, and joint procurement-removal decisions. Through extensive experiments, we show that our approach performs competitively against established benchmarks while naturally learning many structural properties of optimal policies that were previously derived analytically. We introduce a Structure-Informed Policy Network technique that explicitly incorporates these analytical insights into the learning process, enhancing generalization and robustness. Using realistic retail demand data, we demonstrate how this approach helps with extrapolation and provides robustness on out-of-sample data.

Submission Number: 13