LDA-ID: An LDA-based framework for real-time network intrusion detection

Network intrusion poses a severe threat to the Internet. However, existing intrusion detection models cannot effectively distinguish different intrusions with high-degree feature overlap. In addition, efficient real-time detection is an urgent problem. To address the two above problems, we propose a Latent Dirichlet Allocation topic model-based framework for real-time network Intrusion Detection (LDA-ID), consisting of static and online LDA-ID. The problem of feature overlap is transformed into static LDA-ID topic number optimization and topic selection. Thus, the detection is based on the latent topic features. To achieve efficient real-time detection, we design an online computing mode for static LDA-ID, in which a parameter iteration method based on momentum is proposed to balance the contribution of prior knowledge and new information. Furthermore, we design two matching mechanisms to accommodate the static and online LDA-ID, respectively. Experimental results on the public NSL-KDD and UNSW-NB15 datasets show that our framework gets higher accuracy than the others.