Zero-shot reinforcement learning for multi-domain task-oriented dialogue policy

Dialogue policy, a critical component in multi-domain task-oriented dialogue systems, decides the dialogue acts according to the received dialogue state. We introduce zero-shot reinforcement learning for dialogue policy learning, which aims to learn dialogue policies capable of generalizing to unseen domains without further training. This setup brings forward two challenges: 1) the representation of unseen actions & states, and 2) zero-shot generalization to unseen domains. For the first issue, we propose Unified Representation (UR), an ontology-agnostic representation, which effectively infers representations in unseen domains by capturing the underlying semantic relations between unseen actions and states and seen ones. To tackle the second issue, we propose Q-Values Perturbation (QVP), a family of exploration strategies that can be applied either during training or testing. Experiments on MultiWOZ, suggest that UR, QVP, and an integrated framework combining the two are all effective.