Using recurrent neural networks toward black-box system anomaly prediction

Component based enterprise systems are becoming extremely complex in which the availability and usability are influenced intensively by the system's anomalies. Anomaly prediction is highly important for ensuring a system's stability, which aims at preventing anomaly from occurring through pre-failure warning. However, due to the system's complex nature and the noise from monitoring, capturing pre-failure symptoms is a challenging problem. In this paper, we present a sequential and an averaged recurrent neural networks (RNN) models for distributed systems and component based systems. Specifically, we use cycle representation to capture cyclical system behaviors, which can be used to improve prediction accuracy. The anomaly data used in the experiments is collected from RUBis, IBM System S, and the component based system of enterprise T. The experimental results show that our proposed methods can achieve high prediction accuracy with satisfying lead time. Our recurrent neural networks model also demonstrates time efficiency for monitoring large-scale systems.