Large-scale network intrusion detection algorithm based on distributed learning

As Internet bandwidth is increasing at an exponential rate, it's impossible to keep up with the speed of networks by just increasing the speed of processors. In addition, those complex intrusion detection methods also further add to the pressure on network intrusion detection system (NIDS) platforms, and then the continuous increasing speed and throughput of network pose new challenges to NIDS. In order to make NIDS effective in Gigabit Ethernet, the ideal policy is to use a load balancer to split the traffic and forward them to different detection sensors, and these sensors can analyze the splitting data in parallel. If the load balancer is required to make each slice containing all the necessary evidence to detect a specific attack, it has to be designed complicatedly and becomes a new bottleneck of NIDS. To simplify the load balancer, this paper puts forward a distributed neural network learning algorithm. By using the learning algorithm, a large data set can be split randomly and each slice data is handled by an independent neural network in parallel. The first experiment tests the algorithm's learning ability on the benchmark of circle-in-the-square and compares it with ARTMAP (adaptive resonance theory supervised predictive mapping) and BP (back propagation) neural network; the second experiment is performed on the KDD'99 Data Set which is a standard intrusion detection benchmark. Comparisons with other approaches on the same benchmark show that it can perform detection at a high detection speed and low false alarm rate.