A context and semantic enhanced UNet for semantic segmentation of high-resolution aerial imagery

Semantic segmentation of high-resolution aerial images is of paramount importance in a wide range of remote sensing applications. The ever-increasing spatial resolution of aerial imagery brings about two specific challenges that incur labelling ambiguities: intra-class heterogeneity and inter-class homogeneity. To address these two challenges, a novel end-to-end semantic segmentation network for high-resolution aerial imagery, namely Context and Semantic Enhanced UNet (CSE-UNet), is proposed in this paper. Specifically, we exploit multi-level Receptive Field Block (RFB) based skip pathways to enhance the representational power of multi-scale contextual information, and therefore tackle the issue of intra-class heterogeneity. To solve the inter-class homogeneity problem, we propose a dual-path encoder where an auxiliary multi-kernel based feature encoding path is embed to produce strong semantic features at all levels to enlarge the inter-class differences. Experimental results shows that our proposed CSE-UNet achieves competitive performance and outperforms UNet and several other deep networks on the ISPRS Potsdam and Vaihingen datasets.