Performance evaluation of fault diagnosis system based on Bayesian network

Assessing whether a newly developed fault diagnosis system is effective is an important issue to ensure diagnosis system performance. Due to the requirement of evaluating the performance of the fault diagnosis system based on Bayesian network (BN), an evaluation method using a modified binomial distribution was developed, considering the real distribution of diagnosis results. The parameters of the modified binomial distribution were estimated using training data during the training process of fault diagnosis system, and both diagnosis accuracy and confidence interval of a diagnostic system could be calculated simultaneously by this evaluation method. The quantitive evaluation indices provided by the proposed evaluation method greatly contributed to the evaluation of acceptability and reliability of a Bayesian network-based diagnosis system, and were of great significance in supporting diagnosis system training. In conclusion, the effectiveness of the proposed evaluation method was validated by an example concerning a fault diagnosis system for the aircraft fuel system.