Improving autonomous vehicle perception in adverse weather through image augmentation

Robust perception is crucial in autonomous vehicle navigation and localization. Visual processing tasks, like semantic segmentation, should work in varying weather conditions and during different times of day. Semantic segmentation is where each pixel is assigned a class, which is useful for locating overall features. To train a model to perform semantic segmentation, large amounts of data are needed, but the labeling process for segmentation data is especially tedious. Additionally, many large datasets include only images taken in clear weather. This is a problem because training a model exclusively on clear weather data hinders performance in adverse weather conditions like fog or rain. In general, visual processing tasks like segmentation have less accuracy in adverse conditions. We hypothesized that applying image augmentation during training can enhance semantic segmentation performance for autonomous vehicles in adverse weather by decreasing the loss of the model. We trained models with data from Car Learning to Act (CARLA) of both clear images and weather-affected images. Our results show that augmentations significantly improved segmentation in weathered conditions, but models trained on actual weather data still significantly outperformed models trained on augmented data in most conditions. This suggests that while augmentations don’t completely solve the domain adaptation problem, they still offer an improvement.