Log Files Based Prediction of Failure Types using Deep Learning Models

Abstract

In order to verify the correct working of a firmware, a firmware engineer writes various test cases, out of which some tests can get failed due to various issues. Firmware engineer finds the reasons behind the failure of test cases based on logs generated against the test cases, so that they can be resolved and proper verification of firmware can be done. Thus this log files based prediction of a failure types system is developed which is capable of identifying the patterns in the log files that are generated for a test across regression by a firmware engineer and able to classify it in a particular failure type. Error logs are an important source of information both for diagnosis as well as for proactive test failure handling. This log file based prediction includes different stages, namely, Pre-processing, Feature Extraction and Prediction. In Pre-processing phase, regular expressions are written to remove all the non- alphanumeric characters (noise) from logs and to extract test statistics and error description as this is the valuable piece of information or pattern that will help in prediction of failure type. Feature extraction is a process in which we try to extract only that relevant information from the pre-processed data. It can be a low level feature or high level feature. Different classifiers have been used to predict the failure type. This thesis presents a Deep Learning model based prediction system which is trained using Adaptive Moment Estimation (Adam), a Stochastic Optimization algorithm for parameters updation. The Deep Learning models, namely, Convolutional Neural Network (CNN), Recurrent Neural Network-LSTM (RNN-LSTM), and Recurrent Convolutional Neural Network (RNN-CNN) are analogous to neural networks having learnable weights and biases which evaluates log vector to final winner class scores. A comprehensive recognition rate of 87.0% using CNN, 84.0% using RNN-LSTM and 84.0% using RNN-CNN is achieved on a set of 10,000 logs.