The runtime system problem identification method based on log analysis

Currently system logs are an important source of information for system administrators to monitor system behaviors and to identify system problems. The manual examining is infeasible for the complex system and the existing automated methods for identifying system problems have different disadvantages such as the extreme dependency on the source code of the system, the low accuracy of predicting or identifying the system problems, or the requirement of the balanced and labeled training data set. This paper proposes a one- class Support Vector Machine (OCSVM) based method to identify the runtime system problems. Firstly, log sequences are generated for describing the running trajectories of the monitored system by parsing log messages; Secondly, variable length n-gram features are extracted, and moreover, the log sequences are represented as feature vectors based on these variable length n-gram features and Vector Space Model (VSM). Finally, all the feature vectors of the training log sequence set, which only includes the labeled normal log sequences, are input into OCSVM. Experimental results show that it performs better to use linear kernel to train OCSVM on our feature vectors than Gaussian kernel and the size of the sliding window hardly affects the performance of our method. Moreover, the proposed method achieves better performance on unbalanced training dataset than the method based on Random Indexing (RI) and weighted SVM.