An unsupervised real-time tracking and recognition framework in videos

A novel framework for unsupervised face tracking and recognition is built on Detection-Tracking-Refinement-Recognition (DTRR) approach. This framework proposed a hybrid face detector for real-time face tracking which is robust to occlusions, facial expression and posture changes. After a posture correction and face alignment, the tracked face is featured by the Local Ternary Pattern (LTP) operator. Then these faces are clustered into several groups according to the distance between feature vectors. During the next step, those groups which each contains a series of faces can be further merged by the Scale-invariant feature transform (SIFT) operator. Due to extreme computing time consumption by SIFT, a multithreaded refinement process was given. After the refinement process, the relevant faces are put together which is of much importance for face recognition in videos. The framework is validated both on several videos collected in unconstrained condition (8 min each.) and on Honda/UCSD database. These experiments demonstrated that the framework is capable of tracking the face and automatically grouping a serial faces for a single human-being object in an unlabeled video robustly.