Feature selection with partition differentiation entropy for large-scale data sets

Feature selection, especially for large data sets, is a challenging problem in areas such as pattern recognition, machine learning and data mining. With the development of data collection and storage technologies, the data has become bigger than ever, thus making it difficult for learning from large data sets with traditional methods. In this paper, we introduce the partition differentiation entropy from the viewpoint of partition in rough sets to measure the significance and uncertainty of attributes, and present a feature selection method for large-scale data sets based on the information-theoretical measurement of attribute significance. Given a large-scale decision information system, the proposed method first divides it into small sub information systems according to the decision classes. Then by computing partition differentiation entropy in the sub-systems, the partition differentiation entropy of the attribute subset in the original decision information system is obtained. Accordingly, the important features are selected based on the value of partition differentiation entropy. The experimental results show that the idea of the proposed method is feasible and valid.