topic Re: How to extract Specific field and segregate the bunched eventslogs in Splunk Search

How to extract Specific field and segregate the bunched eventslogs

john_snow — Thu, 22 Oct 2020 18:48:35 GMT

I have logs coming from AWS,
first, I need to get just a message (which is an event) from the log
Second, in some logs, we have multiple messages inside log events,
How I can just show logEvents{}.message and segregate the messages from the logs?

Sample log is

{ [-]
   logEvents: [ [-]
     { [-]
       id: 123456789.....
       message: {"Actual Log Event"}
       timestamp: 1601177009988
     }
     { [-]
     }
   ]
   logGroup: CloudTrail
   logStream: 1234567890_CloudTrail_us-east-1
   messageType: DATA_MESSAGE
   owner:1234567890
   subscriptionFilters: [ [-]
   ]
}

Re: How to extract Specific field and segregate the bunched eventslogs

ITWhisperer — Fri, 23 Oct 2020 10:08:48 GMT

| spath input=event logEvents{}.message

This assumes that event contains just the JSON format part of the log.

Re: How to extract Specific field and segregate the bunched eventslogs

john_snow — Wed, 11 Nov 2020 22:11:28 GMT

How I can separate messages from the nested log like in the below log I wanted to separate each message in a log event. We can have single or multiple meesga in a LogEven

{ [-] 
   logEvents: [ [-] 
     { [-] 
       id: 123456789..... 
       message: {"Actual Log Event"} 
       timestamp: 1601177009988 
     } 
     { [-]
       id: 123456789..... 
       message: {"Actual Log Event"} 
       timestamp: 1601177009988 
     } 
   ] 
   logGroup: CloudTrail 
   logStream: 1234567890_CloudTrail_us-east-1 
   messageType: DATA_MESSAGE 
   owner:1234567890 
   subscriptionFilters: [ [-]
   ] 
}

Re: How to extract Specific field and segregate the bunched eventslogs

ITWhisperer — Thu, 12 Nov 2020 09:55:11 GMT

First extract logEvents{}, then extract message from those. Something like

| spath logEvents{} output=logEvents | mvexpand logEvents | spath input=logEvents message

You may need the mvexpand to separate out the different messages.