Subjective quality evaluation of compressed digital compound images

Visual quality evaluation of compressed Digital Compound Images (DCIs) becomes important in many multi-device communication systems. In this paper, we study subjective quality evaluation for compressed DCIs and investigate whether existing Image Quality Assessment (IQA) metrics are effective to evaluate the visual quality of compressed DCIs. A new Compound Image Quality Assessment Database (CIQAD) is therefore constructed, including 24 reference and 576 compressed DCIs. The subjective scores of these DCIs are obtained via visual judgement of 62 subjects using Paired Comparison (PC) in which the Hodgerank decomposition is adopted to generate uncompleted but near balanced pairs. Fourteen state-of-the-art IQA metrics are adopted to assess quality of images in CIQAD, and experimental results indicate that the existing IQA methods are limited in evaluating visual quality of DCIs. Compression results of five coding methods are thus compared with respect to different quality metrics to illustrate the limitation.