Comprehensively evaluating conformance error rates of applying aspect state machines

Aspect Oriented Modeling (AOM) aims to provide enhanced separation of concerns during the design phase and proclaims many benefits (e.g., easier model evolution, reduced modeling effort, and reduced modeling errors) over traditional modeling paradigms such as object-oriented modeling. However, empirical evaluations of these benefits is severely lacking in the AOM community. In this paper, we empirically evaluate one of the AOM profiles: AspectSM, via a controlled experiment to assess if it can help in reducing modeling errors (referred as conformance errors in this paper), which is one of the benefits offered by AOM. AspectSM is a UML profile, which is developed to support automated state-based robustness testing. With AspectSM, crosscutting behaviors are modeled as aspect state machines using the stereotypes defined in AspectSM. We evaluate the conformance error rates of applying AspectSM from various perspectives by conducting four activities: 1) identifying modeling defects, 2) comprehending state machines, 3) modeling state machines, and 4) weaving aspect state machines into base state machines. For most of these activities, experimental results show that the error rates while performing these four activities using AspectSM are significantly lower than standard UML state machine modeling approaches.