Simultaneously Predicting Video Object Segmentation and Optical Flow Without Motion Annotations

Optical flow information is one of the most commonly used temporal cues in video object segmentation algorithms. However, as it is difficult to label real-world video data with motion annotations, video segmentation methods are often forced to use external optical flow datasets and additional flow prediction models. In this paper, we propose an optical flow synthesizing approach which can generate artificial object flow from video segmentation masks, reliving the constraint of manual motion annotations for joint learning of video segmentation and optical flow prediction tasks. Extensive experiments and analysis are carried out on the DAVIS video segmentation datasets and the self-constructed synthetic flow database, demonstrating that the proposed synthetic flow has a better training effect compared with external flow datasets, and that this target-specific flow synthesizing training scheme can help video segmentation networks to better distinguish the motion patterns of certain targets in multiple-instance video segmentation scenes.