topic Re: Context based regex field extraction in Splunk Search

Context based regex field extraction

shrirangphadke — Fri, 19 Jun 2015 00:55:08 GMT

Hi,

I am trying to extract few fields out of logs but Splunk field extraction is not working in my case.

For example:

2015-06-17 13:48:55,689 abc-field [SystemEvent] Time:'June 17, 2015 8:48:55 PM GMT',Severity:'Critical', Event Source:'domain-c0', Code:'301503', Event Message:'Failed to publish abcd configuration version 1408159473758 to cluster domain-c0. Refer logs for details', Module:'abcd something'

2015-06-17 13:48:55,620 abc-xyz-something June 17, 2015 8:48:55 PM GMT INFO SimpleAsyncTaskExecutor-1 SystemEventDaoImpl:124 - [SystemEvent] Time:'June 17, 2015 8:48:55 PM GMT',Severity:'Informational', Event Source:'edge-0', Code:'30101', Event Message:'abcd was booted', Module:'abcd something Appliance'

In above two log snippets I am trying to extract value of the field "Severity".
But since the position of field "Severity" in both the logs are different Splunk returns the field such as:
1. Critical
2. June

Probably it is because Splunk does regex parsing based on position.

I want to extract the fields based on pre-context and post-context.
For example:
Pre-context: "Severity:'"
Required value
Post-context: "', Event"

I am completely stuck here. Please help.

Re: Context based regex field extraction

woodcock — Fri, 19 Jun 2015 01:04:37 GMT

Assuming these are the only 2 variants, try this:

... | rex "^.*?Severity\s*:\s*'?(?<Severity>[^'\s]+)"

Re: Context based regex field extraction

shrirangphadke — Fri, 19 Jun 2015 01:13:26 GMT

Thank you very much for your answer !
Now it will take me another day to understand this 😛

Re: Context based regex field extraction

shrirangphadke — Fri, 19 Jun 2015 01:19:05 GMT

Is there any Splunk regex tutorial which I can follow ?

Re: Context based regex field extraction

woodcock — Fri, 19 Jun 2015 01:23:37 GMT

I learned by doing but that's just the way I am. So although I cannot help you much there, I can suggest some tools. My favorite is Expresso which is free. I use this almost every day. It does a good job of "translating" the RegEx to english on the right side so that when somebody gives you a solution (like I did), it will show you bit-by-bit which each part of the RegEx is doing.

Re: Context based regex field extraction

shrirangphadke — Mon, 22 Jun 2015 20:13:59 GMT

This is exactly what I wanted thank you very much!