Guided scene perception

This thesis addresses the fundamental challenge of depth perception through novel multi-sensor fusion strategies. We introduce a general framework based on virtual pattern projection that seamlessly integrates stereo vision with active sensors, such as LiDARs, mimicking active stereo principles without the limitations of physical pattern projectors. Unlike previous fusion methods, our hallucination approach operates externally to the matching algorithm, making it compatible with any stereo method without code modification or retraining, while achieving state-of-the-art performance with as little as 1\% of active depth measurements. The framework's inherent redundancy enables graceful degradation: in case of active sensor failure, it reduces to passive stereo without accuracy loss; in case of camera failure, it naturally adapts to domain-generalized depth completion -- an unexplored branch of depth completion where a single model, in our case a modern stereo network, can infer dense depth maps across a wide range of environments and conditions. To further reduce setup costs, we propose replacing expensive active sensors with monocular Vision Foundation Models (VFMs). The resulting framework, dubbed Stereo Anywhere, unifies geometric constraints and learned contextual priors within a dual-branch design. This approach achieves zero-shot generalization and remarkable robustness to challenging scenarios such as textureless regions, occlusions, and non-Lambertian surfaces. Furthermore, we extend these principles to the neuromorphic domain, pioneering the fusion of event-based stereo with LiDAR through event hallucination. Finally, we move toward completing the puzzle by extending the Stereo Anywhere concept to the neuromorphic domain. As a first step, we address the absence of event-based Vision Foundation Models by introducing a cross-modal distillation framework that transfers depth priors from RGB-based VFMs to event data, bypassing the scarcity of event data. The next step, integrating monocular cues with the stereo pipeline to build a fully event-based Stereo Anywhere framework, remains a promising direction for future research.