AI-powered resilient perception for autonomous systems

Autonomous systems are critical for safety in self-driving cars, yet their perception layers remain vulnerable to environmental "catastrophic mistakes." While research on weather interference (rain, snow) is growing, the impact of physical sensor-layer contamination—such as mud, dust, oil, and saltwater—remains a dangerous and underexplored reliability gap. This thesis establishes a comprehensive framework for self-reliant perception by addressing the lifecycle of LiDAR degradation. First, we introduce LIDAROC, a novel real-world dataset of 78,000 corrupted frames featuring diverse contaminants—including water, mud, dust, saltwater, lubricant, and foam—applied at multiple severity levels. In addition to controlled settings, the dataset includes an in-situ road-driving subset to capture real-world environmental complexity. Our evaluation reveals that state-of-the-art object detectors (e.g., PointRCNN) suffer significant failures when compromised by physical debris, leading to thousands of high-confidence false detections that training on idealized datasets cannot prevent. To mitigate this, we propose ANZIL, a model-agnostic contaminant detection framework. Utilizing graph-based representations and attention mechanisms, the system acts as an intelligent gating module, identifying corrupted data before it reaches the perception pipeline. Our results show a reduction in catastrophic failures by over 97%, maintaining high sensitivity even in unseen environments. Addressing the challenge of real-time automotive deployment, we utilize knowledge distillation to translate complex Graph Attention Networks into a lightweight student model. This 16-bit quantized 2D-CNN is optimized for RISC-V (GAP8) and NVIDIA Jetson platforms, fitting within 84 KB of memory with a latency of only 26 ms and energy consumption of 210 µJ per inference. Finally, we outline future work toward a few-shot learning framework for real-time adaptation to new contaminants. This work shifts the paradigm from static robustness to continual, self-improving resilience, ensuring autonomous vehicles can reason about their own sensor integrity in the unpredictable real world.