Emphasizing on the timing and type: Enhancing the backchannel performance of virtual agent

Addressing backchannel feedbacks to virtual agent listener gives the agent human-like conversation skills and creates rapport in Human-Computer Interaction. We argue the limitations of current approaches in predicting and generating backchannel. Following two hypotheses emphasizing on the timing and type of backchannel, we introduce an improved system to enhance the agent listener's performance. By using Newcastle Personality Assessor before parasocial consensus sampling and then neural networks, we can obtain the personality rules and select different backchannel timing thresholds for specific agent listener according to its own personality. After a context-free perceptual study, we will build two emotional backchannel lexicons showing positive affection and negative affection respectively. In accordance with the empathy strategy, the system will select one type of backchannel from the corresponding emotional backchannel lexicon. The improved system will be more suitable for different conversation occasions and greatly increase the naturalness between the human speaker and the virtual agent listener in the future.