Response selection with topic clues for retrieval-based chatbots

We consider incorporating topic information into message-response matching to boost responses with rich content in retrieval-based chatbots. To this end, we propose a topic aware attentive recurrent neural network in which representations of the message and the response are enhanced by the topic information. The model first leverages the message and the response represented by recurrent neural networks (RNNs) to weight topic words given by a pre-trained LDA model and forms topic vectors as linear combinations of the topic words. It then refines the representations of the message and the response with the topic vectors through an attention mechanism. The attention mechanism weights the hidden sequences of the message and the response not only by themselves but also by their topic vectors. Thus both the parts that are important to matching and the parts that are semantically related to the topics are highlighted in the representations.Empirical studies on public data and human annotated data show that our model can significantly outperform state-of-the-art methods and rank more responses with rich content in high positions.