Ask a Further Question or Give a List? How Should Conversational Agents Reply to Users’ Uncertain Queries

Conversational agents (CAs) have recently become ubiquitous. Smart speakers, mobile phone voice assistants, and in-car voice assistants have entered our lives. Studies have examined some factors influencing the user experience (UX) of CAs. However, there is little research on CAs’ reply design, especially when the information the users need is uncertain, which is regarded as an uncertain information scenario. The current research mainly focuses on measuring the UX with CAs’ replies in uncertain information scenarios. We designed two reply strategies of CAs, namely, a further inquiry strategy and a list-style reply strategy, to improve UX in this kind of uncertain information scenario. Two studies were designed and conducted based on the E-prime platform to verify the effect of the two reply strategies on UX. In Study 1, we verified the influence of inquiry strategy with different address terms for users on UX. In Study 2, we verified the influence of the list-style reply strategy (explicit and implicit) on UX. The gender differences in the evaluation of the two reply strategies were also examined. The results showed that, in the uncertain information scenarios, the reply strategy of further inquiries received a higher UX evaluation than direct replies in Study 1. Moreover, male participants preferred the “master” address to the “nin” address. However, male participants had no significant preference for further inquiries. For male participants, there was no significant difference in the UX evaluations for the “master” address and “nin” address. List-style replies with ranking information received the highest UX ratings, followed by list-style replies without ranking information and direct replies in Study 2, indicating that users preferred the explicit reply design of CAs. In addition, we found that male participants tended to have a higher rating of CA replies for all reply methods than female participants in our studies, suggesting that women have higher expectations for the reply design of CAs. In general, these results may contribute to the design of CA replies and highlight the importance of personalizing CA language.