topic Re: How can I extract breakable_text properly? in Splunk Search

How can I extract breakable_text properly?

ew09 — Fri, 01 Jul 2016 15:16:30 GMT

Hi everyone,

I have many logs in the following format as an example

Timestamp: 6/27/2016 8:40:25 PM
Message: Matcher: Record not matched for example record
Category: Business
Priority: 10
EventId: 1
Severity: Information
Title:Information
Machine: MachineName
App Domain: Test.exe
ProcessId: 5400
Process Name: E:\Test\Test.exe
Thread Name:
Win32 ThreadId:2600
Extended Properties:

I want to be able to grab all the information after the field name, for example I want a field called Machine, and have the data be 'MachineName.' By using a \n as a delimiter I can split all of the lines up, however I do not want the field name itself. Any advice on what regex to create to do this? Thanks!

Re: How can I extract breakable_text properly?

javiergn — Fri, 01 Jul 2016 15:53:09 GMT

You could try with the extract command at search time:

your search here
| extract pairdelim="\n", kvdelim=":"

Or (better approach probably) try adding the relevant stanzas in props and transforms to extract what you want:

http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Createandmaintainsearch-timefieldextractionsthroughconfigurationfiles

Re: How can I extract breakable_text properly?

ew09 — Fri, 01 Jul 2016 16:15:06 GMT

Thanks for the input. The only issue I run into is on the Message line. There are usually at least two colons, is there a way I can take everything after the Message field even if that character is a colon? Basically, I want to take everything after a colon until it hits a newline

Re: How can I extract breakable_text properly?

javiergn — Fri, 01 Jul 2016 16:49:51 GMT

Another approach using a little bit of regex to get exactly what you need.
You can obviously ignore the first lines that I used to replicate your use case:

| stats count | fields - count
| eval _raw = "
Timestamp: 6/27/2016 8:40:25 PM
Message: Matcher: Record not matched for example record
Category: Business
Priority: 10
EventId: 1
Severity: Information
Title:Information
Machine: MachineName
App Domain: Test.exe
ProcessId: 5400
Process Name: E:\Test\Test.exe
Thread Name: 
Win32 ThreadId:2600
Extended Properties:
"
| rex field=_raw max_match=0 "(?msi)^(?<keyvalue>[^:]+:\s?([^\n]+)?)$"
| mvexpand keyvalue
| rex field=keyvalue "(?i)^(?<key>[^:]+):\s?((?<value>[^\n]+)$)?"
| fillnull value value="NULL"
| fields - keyvalue
| eval {key}=value
| fields - key, value
| stats first(*) as * by _raw

Output: see picture

Re: How can I extract breakable_text properly?

javiergn — Fri, 01 Jul 2016 16:50:08 GMT

see my other answer below.
Hope it helps