Deep multi-view RGB-D fusion for robust and accurate 3D perception

In the last decade, depth sensing has become a prominent technology in fields such as robotics, automotive, mobile, and augmented reality. In such systems an active sensor is employed, i.e., a device exploiting illumination to infer the 3D structure of the framed scene. Time-of-Flight sensors are mainly used indoors on mobile devices, while Light Detection and Ranging sensors are employed in automotive for landscape-like scenarios. Despite the reconstruction accuracy of active depth sensors, their technical limitations demand their integration with other sensors to achieve high accuracy and the technical properties required for specific deployments. This PhD thesis aims to deeply analyze existing technologies for depth estimation and active sensor employment with the ultimate goal of improving and innovating the current technological scenario with novel depth sensing frameworks able to overcome the current limitations. First of all, this is achieved through a detailed analysis of the inherent limitations of active sensors and the proposal of effective solutions. Then, the integration with single or multiple RGB sensors is deeply analyzed in different applicative scenarios, improving the current state of the art with innovative frameworks. Eventually, a fully integrated pipeline able to exploit effectively multi-modal information is proposed.