Learning-driven and distributed optimal control methods for large-scale and multi-agent systems

In recent years, the interest of the control community in large-scale systems has surged, driven by their capacity to encompass behaviors integrating human and cyber-physical resources. The complexity of these systems, characterized by several interconnected units, represents a significant challenge when developing effective optimal control policies. This topic is addressed in this thesis from a threefold perspective: (i) tackling the complex nature of such settings by leveraging data-driven strategies over traditional model-based approaches, (ii) taking advantage of the interconnections typical of these large-scale applications at design level, and (iii) developing efficient and scalable centralized and distributed optimal control algorithms. We start by developing data-driven strategies for the resolution of the Linear Quadratic Regulator problem when the underlying dynamics are unknown. First, we propose an on-policy algorithm, where the system is identified while simultaneously optimizing the control policy. Second, we design data-driven algorithms to generate controllers that adhere to particular structural constraints, such as mirroring the inherent patterns of the large-scale system. Subsequently, we develop a first-order framework for nonlinear optimal control tailored to large-scale systems. This centralized methodology is then extended to the distributed optimal control setting. Here, we tackle a novel problem formulation that leverages the aggregative optimization framework, an approach designed to encapsulate collective behaviors. We then propose a data-driven version of the approach, based on a concurrent optimization and learning scheme. Eventually, the proposed methodology is extended to the stochastic optimal control scenario. Finally, we consider the challenge of developing safe-by-design control policies within the multi-layer control formalism. This framework is tailored to analyzing the control architectures traditionally deployed in robotics applications, where a low-level, easy-to-deploy control policy interacts with a more advanced high-level planning strategy. In this context, we present an optimization-based trajectory generation strategy, ensuring compliance to safety-critical constraints, designed for multi-layer control architectures.