Holistic Correction with Object Prototype for Video Object Segmentation

Recently, memory-based methods have achieved progress in semi-supervised video object segmentation. However, these methods still suffer from unstructured challenges, such as object transformations, occlusions and disappearance-reappearance. To this end, we propose a Holistic Correction Network (HCNet) to adaptively acquire concise object proto­types for holistic correction at semantic, spatial and tempo­ral aspects. Specifically, an Adaptive Prototype Update mod­ule is firstly designed to construct multi-level core object representations by associating object variations in consecu­tive frames with segmentation quality assessment. Based on the updated object prototypes, Semantic, Spatial and Tem­poral Correction modules are respectively designed to en­hance the object semantics in the entire frame, eliminate the incorrect semantic enhancement outside the object re­gions and calibrate the estimated object regions with tempo­ral changes of objects. Through the holistic correction mech­anism with effective object prototypes, our proposed HCNet can robustly and efficiently deal with diverse complex sce­narios. Extensive and comprehensive experiments conducted on seven datasets demonstrate that our proposed HCNet can significantly improve the segmentation performance.