The Effect of Unexpected Visual Stimuli on Short-Term Memory in Immersive Experience

Virtual reality (VR) can offer unexpected and fictional visual content, thus creating unique experiences and satisfying individual imaginations. Nevertheless, the effect of unexpected stimuli on cognition has yet to be studied in VR, and the existing paradigm is unsuitable for dynamic and flexible configurations. This paper investigates the effects of unexpected visual stimuli on short-term memory in VR experience. Building upon typical static and sequential isolation effect studies, we meticulously designed six VR scenes that cover various perceptual patterns, encompassing static or dynamic, as well as sequential, semi-sequential, or non-sequential elements. We then recruit 24 participants to assess short-term memory after each session through two classic goal-directed tasks: free recall and recognition test. To further enhance the credibility of our findings, we record eye-tracking and EEG data to analyze gaze and neurobiological patterns in the presence of unexpected content. The cognitive task results show a significant average recall gain of 30.92%, a recognition accuracy boost of 12.89%, and a shorter reaction time for unexpected items. Moreover, it can observe a longer gaze duration, significant activation in the frontal and parietal regions of the θ and α bands, and an elevation of δ oscillatory energy in the parietal region in response to unexpected visual stimuli. Our study demonstrates that unexpected visual content has a significant impact on human short-term memory and exhibits distinct physiological patterns in VR. The conclusion offers valuable guidance for VR application design and provides a deeper understanding of the consequences of everyday VR use.