Codebook reconstruction with holistic information fusion

Bag of feature model has been shown to be one of the most successful methods in generic image categorisation. However, creating codebook by clustering local feature vectors (e.g. Kmeans) may lose holistic information of images. This study presents a novel process called 'Correlation Feedback' for codebook construction. It introduces semantic similarities of words by measuring correlations among distribution of them within one image. Furthermore, the authors employ label propagation process to spread the affinities among all features. An enhanced codebook is constructed based on fusion of the new similarity matrix with locality preserving projection, which is a linear manifold learning algorithm that can be expanded on both training and testing samples. Experimental results on 15 different scenes and ImageNet show promising performance of importing the novel similarity to dictionary construction.