Anuket Project

Data Mining Based Log Analysis Methods

Intern: Yichen Li 

Weekly Activities:

No. of weekWeek PeriodPlanned ActivitiesCompleted ActivitiesComments
12022/07/18 - 2022/07/22

Technical:

  1. Paper investigation
    1. Read the paper A Survey on Automated Log Analysis for Reliability Engineering
    2. Conclude main procedures and popular techniques used in modern software system log analysis

1a, 1b

Information is concluded in the file Article Review-V0721.pdf
22022/07/25 - 2022/07/29

Technical:

  1. Based on the last few datasets of the paper, sort out the log processing process to see what processing steps are required and the corresponding algorithm application principles.
  2. Investigate more on the specific requirements of datasets, and see whether the existing datasets satisfy the need for research
  3. Research status of several scenarios of log mining and analysis of bottleneck points, and the future trend of log analysis
2, 3Information is concluded in the file Log Analysis Procedure-V0728.pdf
32022/08/01 - 2022/08/051

Due to the lack of efficient source code, the recurrence cannot be fulfilled at present

Information is concluded in the file Log Analysis Procedure-V0804.pdf

42022/08/08 - 2022/08/13

Technical:

  1. Focusing on the HDFS dataset to recurrent the entire processing procedure
  2. Finish investigation of all the methods used in Log Analysis
2Information is concluded in the file Log Analysis Procedure-V0811.pdf
52022/08/15 - 2022/08/19

Technical:

  1. Start recurrency based on HDFS dataset (HDFS dataset from https://github.com/logpai/loghub/) and DeepLog method (https://dl.acm.org/doi/abs/10.1145/3133956.3134015)