Degraded Image Semantic Segmentation Using Intra-image and Inter-image Contrastive Learning

Image degradation increases the difficulty of semantic segmentation, leading to the decreased accuracy. The gap of feature distribution between degraded images and clear images becomes a burden for accurate semantic segmentation. To solve this problem, we propose a contrastive learning method for degraded image semantic segmentation which is parallel to the semantic decoder of semantic segmentation network to utilize both intra-image and inter-image contrastive learning for feature representation. The former forces pixel embeddings and region embeddings belonging to the same semantic class to be more similar than the embeddings belonging to the different classes, while the latter utilizes memory banks to enhance the diversity of positive and negative samples. As a result, the intra-image and inter-image contrastive learning is able to effectively utilize multiple pixels, multiple regions, even multiple images to augment the degraded features and learn a more general representation of degraded features. Experiments demonstrate that our method achieves competitive performance on various benchmarks, including Cityscapes, ADE20K, FoggyDriving and our synthetic datasets. The code and synthetic datasets are open-sourced on Github: https://github.com/cocolord/degraded-image-seg.