Splunk Enterprise

How to get the raw events sorted as per the log available in server?

mnarmada
Path Finder

Hello All,

when I send some log to Splunk, I am not getting the events as per the log order. For example, my first line in the log is 7th or 10th line in Splunk, 2nd line would be 20th and 3rd might me 1st line in Splunk events.

In my data, I need to check whether the file processed successfully or not, any error occurred or not. If processed successfully, how many rows updated. These three things will be in different lines or events. If data is not in correct sorted order, I could not relate one with other. 

Can you please help me to get the data according to the log order. Really appreciate the suggestion!!

Thanks in advance!!

Labels (1)
Tags (4)
0 Karma

ITWhisperer
Ultra Champion

How are you finding out what order the events are in splunk?

Can you share some examples?

0 Karma

mnarmada
Path Finder

Hello @ITWhisperer ,

My log is like below.. From here I need to check whether file committed successfully or any error found. If successfully committed, then how many rows updated.  In splunk, it is configured every line as an event. 

+     1/29/2021 11:20:05 AM : Start of file commit for BSB file A_A__20210128103539.TXT
+     1/29/2021 11:20:07 AM : End of file commit for BSB file A_A__20210128103539.TXT
+     1/29/2021 11:20:07 AM : Successful File Commit for BSB file A_A__20210128103539.TXT
+     1/29/2021 11:20:07 AM : File Summary: 
+     1/29/2021 11:20:07 AM : Row(s) Updated 390
+     1/29/2021 11:20:07 AM : Time Started 29/01/2021 11:20:05 AM
+     1/29/2021 11:20:07 AM : Time Ended 29/01/2021 11:20:07 AM
+     1/29/2021 11:20:07 AM : Reference Date 20210201
+     1/29/2021 11:20:07 AM : OR Code A
+     1/29/2021 11:20:07 AM : OPF Code A

+     1/29/2021 11:19:40 AM : Start of file commit for BSB file B_B__20210128063543.TXT

+     1/29/2021 11:19:45 AM : End of file commit for BSB file B_B__20210128063543.TXT

+     1/29/2021 11:19:45 AM : Error encountered during file commit 70Permission denied

 

I Splunk, I can see like below:

   1/29/2021 11:23:03 AM : TNS OPSG09
+     1/29/2021 11:23:03 AM : FileName A_A__20210129020530.TXT
+     1/29/2021 11:23:03 AM : Path SIF
+     1/29/2021 11:23:03 AM : ProcType D:\SAM\TH\Inbox\BESHEB_TRANSCO\
+     1/29/2021 11:23:03 AM : ValidationKey BESHEB
+     1/29/2021 11:23:03 AM : Start of file commit for BESHEB file A_A__20210129020530.TXT
+     1/29/2021 11:23:03 AM : Special Field 2 
+     1/29/2021 11:23:03 AM : Special Field 1 P2
+     1/29/2021 11:23:03 AM : End of file validation for BESHEB file B_B__20210129020530.TXT
+     1/29/2021 11:22:35 AM : ORCode TH
+     1/29/2021 11:22:35 AM : ConnectionPass SAM_CAD
+     1/29/2021 11:22:35 AM : ConnectionID SAM_CAD
+     1/29/2021 11:22:35 AM : TNS OPSG09
+     1/29/2021 11:22:35 AM : FileName B_B__20210129020530.TXT
+     1/29/2021 11:22:35 AM : Path SIF
+     1/29/2021 11:22:35 AM : ProcType D:\SAM\TH\Inbox\BESHEB_TRANSCO\
+     1/29/2021 11:22:35 AM : ValidationKey BESHEB
+     1/29/2021 11:22:35 AM : Start of file validation for BESHEB file B_B__20210129020530.TXT
+     1/29/2021 11:22:35 AM : Row(s) Updated 856
+     1/29/2021 11:22:35 AM : OPF Code B
+     1/29/2021 11:22:35 AM : OR Code B

So I am not able to map, which row updated for what file.

Please help on this.

Tags (1)
0 Karma

ITWhisperer
Ultra Champion

Since you don't appear to have millisecond timings in your events, but there are at least a number of events with the same second timing, does your application log all the related messages with the same timestamp e.g. all these messages are related?

+     1/29/2021 11:20:07 AM : End of file commit for BSB file A_A__20210128103539.TXT
+     1/29/2021 11:20:07 AM : Successful File Commit for BSB file A_A__20210128103539.TXT
+     1/29/2021 11:20:07 AM : File Summary: 
+     1/29/2021 11:20:07 AM : Row(s) Updated 390
+     1/29/2021 11:20:07 AM : Time Started 29/01/2021 11:20:05 AM
+     1/29/2021 11:20:07 AM : Time Ended 29/01/2021 11:20:07 AM
+     1/29/2021 11:20:07 AM : Reference Date 20210201
+     1/29/2021 11:20:07 AM : OR Code A
+     1/29/2021 11:20:07 AM : OPF Code A

Is there a possibility that unrelated messages could be picked up with the same timestamp? If so, is there a way to distinguish these from each other e.g. do they come from different sources/hosts/files?

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!