How do I edit my transforms.conf to properly extract these fields from my sample data?

akanno — Thu, 18 Feb 2016 04:59:56 GMT

Hi All.

I want to extract fields from the following log data.

headerName=Host, Connection, Accept, headerValue=splunk.com, keep-alive, text/html

I want to extract fields like this.

Host=splunk.com
Connection=keep-alive
Accept=text/html

So I set following in props and transforms

props.conf

[MY_SYSLOG]
REPORT-a = SAMPLE_1,SAMPLE_2,SAMPLE_3

transforms.conf

[SAMPLE_1]
CLEAN_KEYS = 0
FORMAT = $1::$4
REGEX = headerName=(\w+),\s(\w+),\s(\w+),\sheaderValue=(.*?),\s(.*?),\s(.*)

[SAMPLE_2]
CLEAN_KEYS = 0
FORMAT = $2::$5
REGEX = headerName=(\w+),\s(\w+),\s(\w+),\sheaderValue=(.*?),\s(.*?),\s(.*)

[SAMPLE_3]
CLEAN_KEYS = 0
FORMAT = $3::$6
REGEX = headerName=(\w+),\s(\w+),\s(\w+),\sheaderValue=(.*?),\s(.*?),\s(.*)

This setting extracted fields for part log message, but didn't extract fields for the following log message.

headerName=Host, Connection, Accept, Referer, headerValue=splunk.com, keep-alive, text/html, http://google.com

Is there good solution?

Thank you for your help.

Re: How do I edit my transforms.conf to properly extract these fields from my sample data?

maciep — Sun, 06 Mar 2016 17:30:21 GMT

In the example that didn't work, do you want to extract Referer too? Or do you only care about host, connection and accept? Are the values always in the same order after the headerValue= part of the event? Not sure if just hardcoding fieldname is an option or makes sense.

In general though, there are probably a few ways to make this work for you. It's just a regex thing. For example, something like this might work in props.conf

[MY_SYSLOG]
EXTRACT-myfields = headerName=Host,\s+?Connection,\s+?Accept,.+headerValue=(?<host>[^,]+),\s+?(?<connection>[^,]+),\s+?(?<Accept>[^,]+)

topic How do I edit my transforms.conf to properly extract these fields from my sample data? in Splunk Search

How do I edit my transforms.conf to properly extract these fields from my sample data?

Re: How do I edit my transforms.conf to properly extract these fields from my sample data?