Patch-based bag of features for face recognition in videos

Video-based face recognition is a fundamental topic in image processing and video representation, and presents various challenges and opportunities. In this paper, we introduce an efficient patch-based bag of features (PBoF) method to video-based face recognition that plenty exploits the spatiotemporal information in videos, and does not make any assumptions about the pose, expressions or illumination of face. First, descriptors are used for feature extraction from patches, then with the quantization of a codebook, each descriptor is converted into code. Next, codes from each region are pooled together into a histogram. Finally, representation of the image is generated by concatenating the histograms from all regions, which is employed to do the categorization. In our experiments, 100% recognition rate is achieved on the Honda/UCSD database, which outperforms the state of the arts. And from the theoretical and experimental results, it can be derived that, when choosing a single descriptor and no prior knowledge about the data set and object is available, the dense SIFT with ScSPM is recommended. Experimental results demonstrate the effectiveness and flexibility of our proposed method.