K-means method for grouping in hybrid mapreduce clusters

In hybrid cloud computing era, hybrid clusters which are consisted of virtual machines and physical machines become more and more Popular?. MapReduce is a good weapon in this big data era where social computing and multimedia computing are emerging. One of the biggest challenges in hybrid mapreduce cluster is I/O bottleneck which would be aggravated under big data computing. In this paper, we take data locality into consideration and group slave nodes with low intra-communication and high intracommunication. After introducing the architecture and implementation of our grouped hybrid mapreduce cluster (GHMC), we give our k-means algorithm in GHMC and evaluate it with reality environments. The results show that there is a nearly 34.9% performance improvement in our system achieved by the K-means algorithm. Moreover, GHMC system also shows good scalability.