Data augmentation for unbalanced face recognition training sets

Face recognition remains a challenging problem. While one-to-one face verification has been largely tackled, verification-based classification problem still demands effort. To further enhance the verification models, one solution is to fully utilize the unbalanced training sets, where, while abundant samples are provided for some subjects, there are often so few samples available for the rest. These subjects with too few samples can contribute little to the model learning. Therefore, before training a model, algorithms usually perform data augmentation on the whole dataset, especially on subjects with insufficient samples. In this paper, a new augmentation method is proposed, targeting on data augmentation for face classification algorithms. Instead of directly manipulating the input image, we perform virtual sample generating on feature level. The distribution of feature maps is first estimated, then random noise consistent to the distribution is applied to the feature vectors of training samples. Our method is based on Joint Bayesian Face Analysis, and we also develop an algorithm to boost the whole procedure. We conduct experiments based on high dimensional LBP features and features extracted by a shallow Convolutional Neural Network, and succeed to verify the effectiveness of this method, using image data from benchmark dataset LFW.