Perception-Friendly Video Enhancement for Autonomous Driving under Adverse Weather Conditions

Visual perception of an autonomous vehicle is a crucial component of autonomous driving technologies. While visual perception research has achieved promising performance in recent years, modern methods are mostly trained, applied, and tested on single clean images. Recently, deep learning-based perception methods have addressed multiple degrading effects to reflect real-world bad weather cases, but have achieved only limited success, mainly due to 1) less or no temporal information across adjacent frames and 2) poor correlation between image enhancement and visual perception performance. To solve these issues, in this paper we propose a simple and effective video enhancement network incorporating the perception module, one of the high-level vision tasks, which takes video degraded by adverse weather conditions as an input, and produces an enhanced image and a recognition result as output. Besides, this allows us to leverage temporal information across consecutive frames without flow estimation. We also introduce a new training strategy to robustly guide the high-level task model suitable for both high-quality restoration of images and highly accurate perception. Further, considering sequence-level discrimination, we propose a Sequential Contrastive Loss, called SCL, to maximize apparent discrimination over sequential input. By doing this, our algorithm achieves full interaction showing mutual influence between the enhancement and perception tasks. Further, we introduce a novel low memory network dropping out most of the layer connections of dense blocks to reduce memory usage and computational cost while maintaining high performance. Experiment results demonstrate that the proposed method significantly improves the performance on object detection, distance estimation, and memory usage under adverse weather.