Dynamic multi-level attention models for dialogue response generation

One of the key challenges for creating a successful chat bot is to find an effective way to learn from human-human conversation data. Recently, a few neural network based dialog models, including the RNN language model (RNNLM) and the hierarchical recurrent encoder-decoder (HRED) model have shown promising results on dialog response generation. However, there is a critical challenge that the responses generated by these models incline to chit-chat style instead of being informative. In this paper, we empirically investigate this problem and also propose multilevel attention models to extend HRED with a hope that the attention mechanism can capture more informative content. The experiment studies on two multi-turn dialogue Datasets have shown the model’s potential.