TRACE-Net: Hierarchical Feature Matching Network With Thermal Response-Guidance and Adaptive Coherent-Sampling for Endoscopy

Image-guided framewise feature matching in endoscopic surgery is a critical building block enabling precise intraoperative navigation for manual and robotic interventions. However, owing to the inherent textureless and homogeneous nature of cavity inner walls, off-the-shelf feature detection and matching approaches exhibit notable errors or even fail in challenging scenarios with remarkable visual variations. In this work, we propose TRACE-Net, a unified learning-based end-to-end hierarchical feature exploration and matching approach for endoscopic surgery. Specifically, to handle the blurred image input, we introduce an innovative dual-head feature extraction network that simultaneously generates feature descriptors and deblurred feature maps, where the former facilitates thermal response map generation for initial match verification, while the latter implicitly encodes feature information from blurred scenarios. Second, we propose a coherence-informed sampling strategy that leverages depth data and coarse matches to adaptively determine corresponding regions. Besides, we combine the hybrid attention mechanism to integrate the feature descriptor with both intraimage and interimage descriptors. Finally, an expectation-based optimization mechanism is introduced to optimize the matching coordinates for precise point correspondences. We extensively conduct experiments on public, in-house, and clinical surgery sequences, and the results demonstrate that our method significantly outperforms state-of-the-art approaches in both quantitative metrics and qualitative evaluations.