topic Re: Field extraction in cab in Splunk Search

Field extraction in cab

lohit — Tue, 13 Aug 2013 07:13:55 GMT

I have a csv file with following format:

105723,1614:79660877,United States,Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36,,2013-08-06 07:09:49.100000000,92253497

I have done the field extraction on this file delimiting on a comma(,). The problem I am facing is that one of the fields has a comma embedded in it because of which the entries are coming under different column names. Also, this type of log format is not the same in all rows of the csv file. How can I eliminate either (abcd ,efgh), i.e., whole value including the parenthesis or can I ignore the comma between the parenthesis.

Re: Field extraction in cab

kristian_kolb — Tue, 13 Aug 2013 08:17:45 GMT

One way to do it would be to use SEDCMD in props.conf, by which you could remove anything within parentheses, if there is a comma there.

http://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf

Not able to test it at the moment, but something like this ought to work as a starting point;

props.conf (on your indexer, or wherever the parsing phase takes place)

[your_sourcetype]
SEDCMD-drop_bad_comma = s/\s\([^\),]+,[^\)]+\)\s/ /g

Hope this helps,

Re: Field extraction in cab

davecroto — Tue, 13 Aug 2013 08:19:29 GMT

Also, this type of log format is not the same in all rows

Not sure what you mean here. Do you mean the (abcd,efgh) is not always in the same column?

Re: Field extraction in cab

kristian_kolb — Tue, 13 Aug 2013 08:23:57 GMT

not all user-agent strings contain commas.

Re: Field extraction in cab

lohit — Tue, 13 Aug 2013 09:12:58 GMT

@davecroto,kristian

I wanted to say that not in all rows(abcd,efgh) is coming. Sure entries are coming but in some fields delimeted by (;) like (abcd;efgh). I know it is a bad file genrated but this is all i have.