System logs record system states at critical points to help debug failures and promote system stability. Analyzing system logs to detect irregularities establishes more secure and trustworthy systems. Typical log parsing software provides offline, batch processing of raw files, but many applications require constant monitoring not provided by offline methods.
Spell, an online streaming method, parses system event logs to dynamically extract log patterns and maintain a set of discovered message types. DeepLog utilizes Long Short-Term Memory (LSTM) to model a system log as a natural language sequence that automatically learns log patterns. DeepLog detects anomalies when log patterns deviate from the model trained from log data under normal execution. When an anomaly is detected, users can diagnose and perform root cause analysis immediately, thereby increasing system security.