Learnable patchmatch and self-teaching for multi-frame depth estimation in monocular endoscopy

This work delves into unsupervised monocular depth estimation in endoscopy, which leverages adjacent frames to establish supervisory signals during the training phase. For many clinical applications, e.g., surgical navigation, temporally correlated frames are also available at test time. However, most existing monocular methods struggle to make effective use of temporal information during both training and inference, primarily due to the inherent challenges of endoscopic imagery, including low- or homogeneous-texture regions and brightness fluctuations between frames. To fully exploit the temporal information in endoscopic scenes, we propose a novel unsupervised multi-frame monocular depth estimation model. The proposed model integrates a learnable patchmatch module to adaptively increase the discriminative ability in regions with low or homogeneous textures, and enforces cross-teaching and self-teaching consistencies to provide efficacious regularizations towards brightness fluctuations. Furthermore, as a byproduct of the self-teaching paradigm, the proposed model is able to improve the depth predictions when more frames are input at test time. We conduct detailed experiments on multiple datasets, and the experimental results indicate that the proposed method exceeds prior state-of-the-art competitors. The source code and trained models will be publicly available at https://github.com/ShuweiShao/FrameDepth.