Online Detection of Domain-Specific New Words in Text Streams

With the tremendous development of Internet, many domain-specific new words appear in various media text streams such as forums, Sina Weibo, Wechat, etc. These new words are always a group of important words in specific domains and are significant for NLP tasks. Most existing models have time-consuming processing or cannot handle out of vocabulary (OOV) words on streaming and online scenes. In this paper, we propose an unsupervised method, D-TopWords with Gaussian LDA, to perform online detection of domain-specific new words effectively. Different from traditional new words detection models, our method is a joint statistical model based on a finite word dictionary without any handcraft features. By further introducing Gaussian LDA into our model, we solve properly the problem of OOV words from new text streams. Experimental results show that our work can successfully extract domain-specific new words and it has a better performance in online detection task than some state-of-the-art methods.