topic Can I define fields like in AWK, basically define extract field without using regex. in Splunk Search

Can I define fields like in AWK, basically define extract field without using regex.

clyde772 — Fri, 07 May 2010 23:23:02 GMT

Let's say we want to process the typical data input like below :

12|Jones Indiana|76|223-33-3323|US|CALIFORNIA|MARRIED

In splunk, I have to use "rex" and do a whole bunch of regex to parse out the fields. Is there a way in Splunk to process these kind of structured log like in awk manner?

awk manner, meaning

awk -F"|" '{print $1" "$2" "$3}'  and so on...

In another words define pattern for delimiter which is "|" and just assign values with field number like $1, $2

I thought powerful engine like splunk would have a similar way to process. Field parsing without doing while bunch of regex.

YhC.

Re: Can I define fields like in AWK, basically define extract field without using regex.

Simeon — Fri, 07 May 2010 23:31:31 GMT

You can use the DELIMS parameter to extract fields in that manner. For example, we use the following for csv files:

[extract_csv]
DELIMS = ","
FIELDS = "field1", "field2", "field3"

To correctly extract the fields in this manner, you should review the following page which details how to configure complex extractions through configuration files:

http://www.splunk.com/base/Documentation/latest/Knowledge/Createandmaintainsearch-timefieldextractionsthroughconfigurationfiles

Re: Can I define fields like in AWK, basically define extract field without using regex.

Dan — Sat, 08 May 2010 06:23:06 GMT

You could also use the | extract command at search time. It takes a delims parameter.

Re: Can I define fields like in AWK, basically define extract field without using regex.

gkanapathy — Mon, 10 May 2010 12:33:24 GMT

This is sometimes useful, but the extract command's delims is actually a pair pairdelim and kvdelim which are both required. Splunk doesn't generate sequential names like this.

Re: Can I define fields like in AWK, basically define extract field without using regex.

gkanapathy — Mon, 10 May 2010 12:34:42 GMT

This has very little to do with power, and more to do with clarity. While it's undoubtedly convenient to be able to throw down one-line expressions, the intention with Splunk is usually to define and name fields meaningfully for shared and long-term re-use.