Fast intra-prediction mode decision algorithm for high efficieny video coding

In this paper, we propose a fast intra-prediction mode decision algorithm based on the probability statistics of the best prediction mode and the relevance of the neighboring blocks for high efficiency video coding (HEVC) to reduce its encoding complexity. The intra prediction candidates are reduced before Rate Distortion Optimization (RDO) procedure by two steps of rough mode decision. Sum of Absolute Difference (SAD) of the residuals is used in the first step and modes with smaller SAD values are retained. We execute the Hadamard transform on prediction residuals from the rough modes to derive the Sum of Absolute Hadamard Transformed Difference (SATD), and then the rate-distortion cost function based on SATD is used in the second rough mode decision. Finally the most probable mode (MPM) is considered for RDO process to select the best intra prediction mode. Experiments results demonstrate that our proposed algorithm is quite efficient for intra prediction speedup. Compared with HM 4.0, the proposed algorithm reduces the encoding time by 30.18% while the PSNR decrease is negligible, 0.06 dB on average. And the bitrate is only increased by 1.97% on average.