A Novel Keypoint-Based Approach for Occlusion Problem in Cross-View Labelling

Labelling multiple people across various camera views is fundamentally challenging, because of severe occlusion, complex scenes, and changing visibility observed from various viewpoints. In this paper, we propose a novel approach to solve the occlusion problem for cross-view labelling based on pairs of images. To solve the multiple people matching problem while severe occlusion occurs, we introduce a novel keypoint-based correspondence component between different people, in order to overcome the mis-alignment of multi-view geometry and locate the proper cross-view correspondence when occlusion occurs. Based on the keypoint component, correct correspondence can be rectified, thus obtaining accurate multi-view distance. To further enhance the performance for cross-view labelling, we incorporate pre-trained appearance features and multi-view constraints to achieve multiple people matching. Pairwise score strategy is also applied to increase the labelling accuracy. Experimental results verify the superior performance of the proposed method on challenging multi-view multiple people datasets both quantitatively and qualitatively.