When a GUI regression test failed, what should be blamed?

Jin Chen; Mengxiang Lin; Kai Yu; Bing Shao

doi:10.1109/ICST.2012.127

When a GUI regression test failed, what should be blamed?

Jin Chen^*
, Mengxiang Lin
, Kai Yu
, Bing Shao

^*此作品的通讯作者

机械工程及自动化学院

Beihang University

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

5 引用（Scopus）

摘要

Script-based automated regression testing is widely used in industry. In this work, we focus on failed tests in a real regression test project. The causes of 197 failed tests produced in automated testing are examined and categorized based on an analysis procedure presented. The result shows that incorrect scripts, oracle mismatches, test tool bugs and misconfigurations involved in testing contribute most to failures instead of product bugs. Detecting and fixing the false positives are laborious and time-consuming. Our empirical study demonstrates that the benefits of test automation are obliterated to some extent in practical settings. To improve the effectiveness of automated regression testing further, some issues should receive more attention.

源语言	英语
主期刊名	Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012
页	467-470
页数	4
DOI	https://doi.org/10.1109/ICST.2012.127
出版状态	已出版 - 2012
活动	5th IEEE International Conference on Software Testing, Verification and Validation, ICST 2012 - Montreal, QC, 加拿大期限: 17 4月 2012 → 21 4月 2012

出版系列

姓名	Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012

会议

会议	5th IEEE International Conference on Software Testing, Verification and Validation, ICST 2012
国家/地区	加拿大
市	Montreal, QC
时期	17/04/12 → 21/04/12

访问文件

10.1109/ICST.2012.127

其它文件与链接

链接到 Scopus 的出版物

引用此

Chen, J., Lin, M., Yu, K., & Shao, B. (2012). When a GUI regression test failed, what should be blamed? 在 Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012 (页码 467-470). 文章 6200139 (Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012). https://doi.org/10.1109/ICST.2012.127

@inproceedings{d229f0c338424a4daad02eb7c328b6ea,

title = "When a GUI regression test failed, what should be blamed?",

abstract = "Script-based automated regression testing is widely used in industry. In this work, we focus on failed tests in a real regression test project. The causes of 197 failed tests produced in automated testing are examined and categorized based on an analysis procedure presented. The result shows that incorrect scripts, oracle mismatches, test tool bugs and misconfigurations involved in testing contribute most to failures instead of product bugs. Detecting and fixing the false positives are laborious and time-consuming. Our empirical study demonstrates that the benefits of test automation are obliterated to some extent in practical settings. To improve the effectiveness of automated regression testing further, some issues should receive more attention.",

keywords = "GUI testing, automated testing, regression testing, test maintenance",

author = "Jin Chen and Mengxiang Lin and Kai Yu and Bing Shao",

year = "2012",

doi = "10.1109/ICST.2012.127",

language = "英语",

isbn = "9780769546704",

series = "Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012",

pages = "467--470",

booktitle = "Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012",

note = "5th IEEE International Conference on Software Testing, Verification and Validation, ICST 2012 ; Conference date: 17-04-2012 Through 21-04-2012",

}

Chen, J, Lin, M, Yu, K & Shao, B 2012, When a GUI regression test failed, what should be blamed? 在 Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012., 6200139, Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012, 页码 467-470, 5th IEEE International Conference on Software Testing, Verification and Validation, ICST 2012, Montreal, QC, 加拿大, 17/04/12. https://doi.org/10.1109/ICST.2012.127

When a GUI regression test failed, what should be blamed? / Chen, Jin; Lin, Mengxiang; Yu, Kai 等.
Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012. 2012. 页码 467-470 6200139 (Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012).