TTCN-3 based robustness test generation and automation

Robustness testing is a widely accepted approach to detect potential robustness weaknesses. However, the existing robustness testing approach suffers from some limitations, especially lacking support for automatic extraction of the SUT (System Under Test) specification knowledge, such as input syntax and interaction scenario. Such information is crucial for robustness test generation. In this paper, we present a TTCN-3 based robustness test generation and automation approach. The approach at first extracts the input syntax of SUT from TTCN-3 functional test suite; then generates invalid inputs according to the input syntax; at last automates the test execution by reusing the interaction scenarios defined in functional test cases to achieve invalid input injection and test verdict determination. We conducted a case study on three widely used SIP terminals. In the case study, our approach detected several different robustness problems in all three SIP terminals, and some of them are critical.