Crowdsourcing Reinforcement Learning to Optimize Knee Replacement Pathway

Hao Lu, Mengdi Wang

Keywords: clinical pathway optimization, reinforcement learning, knee replacement, medical claims

Abstract: Joint replacement is the most common inpatient surgical treatment in the US. We investigate the clinical pathway optimization for knee replacement, which is a sequential decision process from onset to recovery. Based on episodic claims from previous cases, we view the pathway optimization as an intelligence crowdsourcing problem and learn the optimal decision policy from data by imitating the best expert at every intermediate state. We develop a reinforcement learning-based pipeline that uses value iteration, state compression, kernel representation and cross validation to predict the best treatment policy. It also provides forecast of the clinical pathway under the optimized policy. Empirical validation shows that the optimized policy achieves 7% reduction of the overall cost and 33% reduction of the excessive cost premium.