Cloth-Changing Person Re-identification with Human Keypoint Prediction

Cloth-changing person re-identification is a more challenging yet practical ReID task, where one key idea is to additionally leverage appearance-invariant biological features (e.g., face/hair style, body structure or gait). Most current works focus on aggregating structural information (e.g., contour) with appearance feature (e.g., color and texture) through fusion or embedding strategies. In this paper, we propose a multi-task ViT-Heatmap network that incorporates human keypoint. Specifically, we use identity labels to supervise cloth-sensitive appearance feature learning in the appearance branch, while simultaneously using keypoint heatmaps as structural information to supervise identity-relevant feature learning in the heatmap branch. In this way, the network is guided to focus on human body structure and explicitly learn appearance-invariant features. We conduct experiments on five cloth-changing ReID datasets (i.e., DeepChange, LTCC, PRCC, LaST, and Celeb-ReID). The experimental results show that our ViT-Heatmap method achieves state-of-the-art performance in cloth-changing scenarios.