Boundary-Aware Multitask Learning for Remote Sensing Imagery

Semantic segmentation and height estimation play fundamental roles in the scene understanding of remote sensing images with their wide variety of aerial applications. Recently, deep convolutional neural networks (DCNNs) have achieved state-of-the-art performance in both tasks. However, DCNN-based methods learn to accumulate contextual information over large receptive fields while lose the local detailed information, resulting in blurry object boundaries. The complicated ground object distribution and low interclass variance further aggravate the difficulty in generating accurate predictions. To address the above-mentioned issues, we propose a novel boundary-aware multitask learning (BAMTL) framework to perform three tasks, semantic segmentation, height estimation, and boundary detection, within a unified model. The boundary detection is employed as an auxiliary task to regularize the other two master tasks at both the feature space and output space. We present a boundary attentive module to build the cross-task interaction for master tasks, which enforce the networks to filter out the confident area and focus on learning the high-frequency details. We then introduce a boundary regularized loss term to further refine the prediction maps to be locally consistent while preserving boundary structures. With these formulations, our model improves the performance of both segmentation and height tasks, especially along the boundaries. Experimental results on two publicly available remote sensing datasets demonstrate that the proposed approach performs favorably against the state-of-the-art methods.