Reinforcement learning for dynamic resource allocation in distributed systems

Resource Allocation (RA) problems are ubiquitous across diverse domains, spanning from health services provisioning, supply chain management, personnel scheduling as well as task scheduling, cloud resources orchestration, and network capacity allocation. When the resource request changes over time, we refer to the field of Dynamic Resource Allocation (DRA) problems. DRA problems find applicability in domains such as cloud computing, network management, energy or water supply, and public transportation systems as they allow adjustment of the number of resources assigned for each component or user. However, conventional allocation strategies employed in DRA yield suboptimal policies, primarily due to their reactive nature. They tend to overlook the long-term implications of allocations over future time windows. This thesis proposes an innovative approach to tackle DRA problems by leveraging reinforcement learning (RL). Through trial and error, RL allows the learning of policies that proactively consider the effects of allocations over time and respond effectively to unexpected changes in demand. The effectiveness and efficiency of this proposed approach are supported by empirical evaluations on challenging DRA problems. Initially, it addresses task scheduling in heterogeneous worker-based distributed queues, integrating an adaptive RL method with the popular Celery task queuing system. Subsequently, the Thesis presents a deep reinforcement learning (DRL) based resource orchestrator tailored for managing virtual resources in Open Radio Access Network (O-RAN) infrastructures. Finally, it implements a DRL solution for efficient bike redistribution in Bike Sharing Systems (BSS), simultaneously optimizing operational costs for system operators. The results, derived from both synthetic and real-world data, underscore the superiority of RL approaches over greedy allocation strategies, demonstrating enhanced optimization of the specified objectives. Moreover, the thesis provides technical insights on how to efficiently design and implement these solutions in real-world and well-engineered prototypes.