Uncertainty-Wise Cyber-Physical System test modeling

It is important that a Cyber-Physical System (CPS) with uncertainty in its behavior caused by its unpredictable operating environment, to ensure its reliable operation. One method to ensure that the CPS will handle such uncertainty during its operation is by testing the CPS with model-based testing (MBT) techniques. However, existing MBT techniques do not explicitly capture uncertainty in test ready models, i.e., capturing the uncertain expected behavior of a CPS in the presence of environment uncertainty. To fill this gap, we present an Uncertainty-Wise test-modeling framework, named as UncerTum, to create test ready models to support MBT of CPSs facing uncertainty. UncerTum relies on the definition of a UML profile [the UML Uncertainty Profile (UUP)] and a set of UML Model Libraries extending the UML profile for Modeling and Analysis of Real-Time and Embedded Systems (MARTE). UncerTum also benefits from the UML Testing Profile V.2 to support standard-based MBT. UncerTum was evaluated with two industrial CPS case studies, one real-world case study, and one open-source CPS case study from the following four perspectives: (1) Completeness and Coverage of the profiles and Model Libraries in terms of concepts defined in their underlying uncertainty conceptual model for CPSs, i.e., U-Model and MARTE, (2) Effort required to model uncertainty with UncerTum, and (3) Correctness of the developed test ready models, which was assessed via model execution. Based on the evaluation, we can conclude that we were successful in modeling all the uncertainties identified in the four case studies, which gives us an indication that UncerTum is sufficiently complete. In terms of modeling effort, we concluded that on average UncerTum requires 18.5% more time to apply stereotypes from UUP on test ready models.