A framework of automatic testing of image processing applications

Image processing applications are playing indispensable role in the modern life. These applications are required to be reliable due to their significance and human dependence. However, testing of these applications is very challenging due to complex nature of images also carrying visual semantics. Currently, the testing of these applications is performed manually, carried out by selecting some standard test images to exercise the implementation under test and their output images are evaluated by a human oracle. Manual selection, execution and evaluation of images have some limitations such as, a tedious job, selection of appropriate test images, unavailability of a domain expert, different expert levels of the evaluators, limited number of test cases, coverage evaluation etc. The increasing complexity of image processing applications and limitations of manual testing has elicited the requirement of automatic testing of image processing applications. In this paper, we have proposed a fully automatic framework to test image processing applications which includes generation of test images, execution of generated test cases and correctness evaluation of output images. In our previous work, we proposed a novel image generation technique based on symbolic evaluation of program under test and also proposed two schemes to address test oracle problem, using metamorphic testing and support vector machines. The proposed framework integrates test image generation, tests execution, effectiveness measurement of test cases, stopping criteria and output evaluation. We demonstrated our approach by experiments and the preliminary results show that the proposed framework is useful to eliminate tedious manual work and enables more effective testing with automatic input generation and output evaluation.