A mathematical modeling framework for software reliability testing

Software reliability testing refers to various software testing activities that are driven to achieve a quantitative reliability goal given a priori or lead to a quantitative reliability assessment for the software under test. In this paper we develop a modeling framework for the software reliability testing process, comprising a simplifying model and a generalized model. In both models the software testing action selection process and the defect removal mechanism are explicitly described. Both the discrete-time domain and the continuous-time domain are involved. The generalized model is more accurate or realistic than the simplifying model since the former avoids the assumption that defects are equally detectable and the assumption that defects are removed upon being detected. However simulation examples show that the simplifying model really captures some of essential features of the software testing process after a short initial testing stage. The modeling framework is practically realistic, mathematically rigorous, and quantitatively precise. It demonstrates that the relationship between software testing and delivered software reliability, which was poor understood, can well be formulated and quantified. Rigorous examinations show that several common assumptions adopted in software reliability modeling, including the independence assumption, the exponentiality assumption, and the NHPP assumption, are theoretically false in general. This paper sets a good starting point to further formalize and quantify the software testing process and its relation to delivered software reliability.