Hi Team,
Would like to design the query for the below requirement where we wanted to capture 2 dash boards as below for the information we have as logs (refer test data in this attachment which is the sample log data)
Dash board1 – To get details of the logs based on the field Thread_number and Application_MSG having APP Request (which is part of my incoming log data). This is designed already.
Timestamp1 – Combination of Thread_number and Application_MSG having APP Request occurnace
Timestamp2 - Combination of Thread_number and Application_MSG having APP Request next occurance and to be continued for each occurrence
Dash board2 – This is a drill down dashboard from Dashboard1 which has to return the list of lines between the rows displayed in Dashboard1
Test Data :
2018-05-14 14:25:00,093 INFO STDOUT 14:25:00,093 [com.xxx] INFO - APP Request [ RequestInformation1 ]
2018-05-14 14:25:00,108 INFO com.777 Transform - Completed server response transform. Took 31 ms.
2018-05-14 14:25:10,189 INFO com.777 Listing exeuction. Took 32 ms.
2018-05-14 14:25:12,109 INFO com.777 Query processed.
2018-05-14 14:25:13,112 INFO com.777 Query processed.
2018-05-14 14:25:14,053 INFO STDOUT 14:25:00,093 [com.xxx] INFO - APP Request [ RequestInformation2 ]
2018-05-14 14:25:13,124 INFO com.777 Response processed.
Fields
AUDIT_TIME, LOGGING_PRIORITY, LOG_LEVEL, CONNECTION_FACTOR ,THREAD_NUMBER ,,AUDIT_DATA
Regular Expression
^(?P<AUDIT_TIME>[^,]+)[^,\n]*,(?P<LOGGING_PRIORITY>\d+)\s+(?P<LOG_LEVEL>\w+)\s+(?P<CONNECTION_FACTIRY>[^ ]+)\s+(?P<THREAD_NUMBER>[^ ]+)\s+(?P<AUDIT_DATA>.+)
AUDIT_TIME: 2018-05-14 14:25:00
Logging_Priority : 331
Log_Level : INFO
Connection_factory : [STDOUT]
Thread_Number : (http-123.123.123-800-8)
AUDIT_TIME: 14:25:00,093 [com.xxx] INFO - APP Request [ RequestInformation1 ]
Dashbaord1 shows
2018-05-14 14:25:00,093 INFO STDOUT 14:25:00,093 [com.xxx] INFO - APP Request [ RequestInformation1 ]
2018-05-14 14:25:14,093 INFO STDOUT 14:25:00,093 [com.xxx] INFO - APP Request [ RequestInformation2 ]
Expected output in dash board2
2018-05-14 14:25:00,093 INFO STDOUT 14:25:00,093 [com.xxx] INFO - APP Request [ RequestInformation1 ]
2018-05-14 14:25:00,108 INFO com.777 Transform - Completed server response transform. Took 31 ms.
2018-05-14 14:25:10,108 INFO com.777 Listing exeuction. Took 32 ms.
2018-05-14 14:25:12,108 INFO com.777 Query processed.
2018-05-14 14:25:13,108 INFO com.777 Query processed.
2018-05-14 14:25:14,093 INFO STDOUT 14:25:00,093 [com.xxx] INFO - APP Request [ RequestInformation2 ]
2018-05-14 14:25:13,108 INFO com.777 Response processed.
In the above data we have mapped THREAD_NUMBER: http-123.123.123-800-8 as value A and AUDIT_DATA: "APP Request" as value B
So now I’m trying to print the below events as they are the values between the same combination values of A & B (i.e. THREAD_NUMBER: http-123.123.123-800-8 & AUDIT_DATA: "APP Request")
2018-05-14 14:25:00,093 INFO STDOUT 14:25:00,093 [com.xxx] INFO - APP Request [ RequestInformation1 ]
2018-05-14 14:25:00,108 INFO com.777 Transform - Completed server response transform. Took 31 ms.
2018-05-14 14:25:10,108 INFO com.777 Listing exeuction. Took 32 ms.
2018-05-14 14:25:12,108 INFO com.777 Query processed.
2018-05-14 14:25:13,108 INFO com.777 Query processed.
