Learn From Past to Future: Exploiting Self-Training and Curriculum Learning in Remote Sensing Class-Incremental Semantic Segmentation

Class-incremental semantic segmentation focuses on updating the segmentation model with only new-class samples. Catastrophic forgetting and background shift are the two prevalent challenges. We identify two additional issues in remote sensing data that worsen these problems: significant class distribution variability and error accumulation-induced model degradation. To solve these three problems, we propose a new self-training and curriculum learning guided dynamic refined network (STCL-DRNet). First, we introduce a self-training auxiliary branch to complement the frozen last-step model, integrating cross-step knowledge to mitigate rapid forgetting. Then, a gradient-oriented dynamic refined loss is proposed to assess under-learned classes and mitigate class imbalance. Furthermore, class-balanced curriculum learning (CL) is embedded to alleviate performance degradation throughout incremental training. Extensive experiments on benchmark datasets, including DeepGlobe, iSAID, ISPRS Potsdam, and Vaihingen, demonstrate that the proposed STCL-DRNet achieves state-of-the-art (SOTA) performance. In the 1-1s setting of the DeepGlobe dataset, STCL-DRNet exceeds previous SOTA methods by 11.6% in mIoU. For the iSAID 10-1s setting, it outperforms the previous SOTA by 12.76% in mIoU. As for ISPRS Potsdam and Vaihingen, our STCL-DRNet surpasses the SOTA by 5%–8% in all settings. Visualization and analysis further validate its interpretability.