Optimizing a Multi-echelon, Lost-Sales Inventory Management System through Deep Reinforcement Learning

This research investigates the intricate challenges within multi-echelon multi-product lost-sales inventory management, aiming to streamline complexities using Deep Reinforcement Learning (DRL) principles. The study addresses a lost-sales inventory management problem spanning multiple echelons and products, aiming to maximize network profit while determining optimal quantities from upstream suppliers. Informed by notable literature emphasizing the efficacy of DRL in optimizing inventory strategies, this study focuses on applying DRL specifically to the Multi-Product Multi-Echelon context.

We begin by constructing a complex inventory management model accommodating lost sales and intricate cost structures. Employing DRL, we develop a framework to tackle this problem and evaluate its efficacy. Our investigation extends to various scenarios, encompassing changes in lead time, fixed and sourcing costs, holding expenses, lost sales penalties, and demand variations.

The overarching goal is to tackle the complexities associated with lost sales and nuanced profit structures within the framework of multi-product multi-echelon inventory management. By doing so, this research contributes to advancing decision-making methodologies using DRL in this expansive state-action space. Additionally, we aim to share initial results of applying DRL in this operational problem and address the research question: Does deep reinforcement learning produce superior policies for the multi-echelon, multi-product, lost-sales inventory management problem, in comparison with standard reinforcement learning?

Fatemeh Fakhredin started her PhD program at Kühne Logistics University in September 2022 under the primary and secondary supervision of Prof. Dr. Joern Meissner and Prof. Dr. Asvin Goel. Her research focuses on machine learning and its application in the fields of logistics and supply chain.

Fatemeh did her Bachelor of Science in Industrial Engineering (IE) at Amirkabir University of Technology (AUT), focusing on Data Mining and Operations Research approaches. She has also achieved “The Best Student Award” from the IE department of AUT for four consecutive years. Furthermore, she finished her bachelor’s as a 2nd-ranked student among all of her co-faculty students in 2016. Since these achievements and as a “brilliantly talented” student, she was admitted to peruse her M.Sc. education straightforwardly, without taking any entrance exam at AUT and she was granted a scholarship from Iran’s National Elite Foundation (INFE) in 2016 and 2017. During her Master of Science in Industrial Engineering (the specific field of Systems Optimization) at AUT, she set her focus on developing a two-stage stochastic non-linear model for dynamic ride-sharing, which is a kind of vehicle routing problem. Her master thesis was “Optimizing the Dynamic Ride-Sharing to Reduce the Traffic Jam in Large Cities: Case Study - Implementation of Ride-Sharing in Tehran City”. During her studies, Fatemeh gained practical experience in the fields of transportation, closed-loop supply chain, and green supply chain with publishing a paper in 2017. She graduated with her master’s degree as a 1st-ranked student in 2018.

Before joining KLU, to gain practical experience besides her academic background, Fatemeh worked as a “System Analyst and System Developer” at PSP Company (a subdivision of Golrang Industrial Group) in Iran. Her main duties were extracting, optimizing, and modeling processes in BPM-Software (BPMS) to improve the organization’s efficiency and automate tasks for a better customer experience. She also extended her knowledge about data science by learning and working with SQL Server and Python.