Abstract

In many operations management problems, we need to make decisions sequentially to minimize the cost while satisfying certain constraints. One modeling approach to study such problems is constrained Markov decision process (CMDP). When solving the CMDP to derive good operational policies, there are two key challenges: one is the prohibitively large state space and action space; the other is the hard-to-compute transition kernel. In this work, we develop a sampling-based primal-dual algorithm to solve CMDPs. Our approach alternatively applies regularized policy iteration to improve the policy and subgradient ascent to maintain the constraints. Under mild regularity conditions, we show that the algorithm converges at rate  where T is the number of iterations. When the CMDP has a weakly coupled structure, our approach can substantially reduce the dimension of the problem through an embedded decomposition. We apply the algorithm to two important applications with weakly coupled structures: multi-product inventory management and multi-class queue scheduling, and show that it generates controls that outperform state-of-art heuristics.

Time

12月1日（星期二）09:30-11:00

Speaker

Yi Chen is a fifth-year Ph.D. student at Northwestern University, Department of Industrial Engineering and Management Sciences. His research interests are at the interface of operations research and data science. His current research primarily focuses on developing advanced algorithms with provable efficiency to facilitate better operational decision-making. He is also interested in studying consumer behavior via machine learning and econometrics tools and aims to understand the implications of customer behavior on operational strategies.

Venue

ZOOM会议ID：95984622667

参会密码：696442