A discrete event simulation and deep reinforcement learning framework for queue management and patient prioritization in an emergency department

The Emergency Department (ED) represents one of the main access points to hospital care. Its complexity derives from the intrinsic high operational variability, related to the non-elective nature of the services provided. One of the most challenging and widely documented problems in this setting is overcrowding, which occurs when the demand for emergency services exceeds the capacity to provide them in a timely manner. Overcrowding, directly correlated with prolonged patient length of stay (LOS), is also strongly associated with reduced quality of care, discomfort and distress for both patients and staff. This thesis proposes a data-driven operational decision support tool for mitigating overcrowding through adaptive patient flow management. The approach combines Discrete Event Simulation (DES), Deep Learning (DL) and Deep Reinforcement Learning (DRL) within a unified framework, and it is applied to the case study of the General ED of IRCCS Azienda Ospedaliero-Universitaria di Bologna, a large hospital in Northern Italy. First, a DES is developed and validated as a Digital Twin of the real ED, fed with historical data that allow a realistic and dynamic representation of the system. The patient prioritization problem is then formulated as a Markov Decision Process (MDP), with the objective of reducing LOS while ensuring timely access to care. The MDP formulation is further enriched through a DL model that estimates the likelihood of each patient approaching the end of their clinical pathway. Two Actor-Critic models are trained and evaluated within the simulated environment, and their performance is compared against the prioritization policy currently adopted in the real ED. The results indicate a reduction in both LOS and crowding levels, while taking into account clinical constraints. These findings demonstrate the feasibility of adopting similar approaches to improve ED patient flow, opening a promising research direction towards data-driven and adaptive control of emergency operations.