A fast HEVC intra mode decision algorithm based on machine learning

In view of the high computational complexity of high efficiency video coding (HEVC) encoding, a fast algorithm based on machine learning is proposed in this paper. According to the smoothness of image content, we divide prediction units (PUs) into three classes. The smooth PU has no need to test all the intra prediction modes. Thus, the computational complexity of the algorithm can be reduced effectively. First, we calculate the variance of the reference pixels on the left side, the above side of each PU, and the variance of all the reference pixels, as well as the optimal intra prediction mode for each PU. The variances reflect the smoothness of the reference pixels. Then, the machine learning software of Weka is used to classify the obtained data previously, and a decision tree is generated. Finally, according to the decision tree, the intra prediction modes for each PU to be tested are determined, then these intra modes are tested for each PU to choose the optimal mode, reducing unnecessary process, thus reducing the encoding complexity. Experimental results show that compared with the standard HEVC 15.0 coding algorithm, in the case of high bitrate, the encoding time is reduced by about 16.18% on average with negligible increase of Bjontegaard delta rate (BD-rate) (about 0.25%) and decrease of Bjontegaard delta peak signal-to-noise rate (BD-PS)NR (about 0.02 dB).In the case of low bitrate, the encoding time is reduced by about 20.75% on average with negligible increase of BD-rate (about 0.04%) and decrease of BD-PSNR (about 0.00 dB).