topic Re: How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports? in Splunk Search

How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

tlbj6142 — Fri, 23 Jan 2015 18:41:37 GMT

Most of the time we use a shared report ("General Product Report") to view our logs for sourcetype="product". I created a field extraction rule to parse each entry into 7-8 fields (the sample below has been trimmed down for brevity).

^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s(?P<body>.*)$

I'd like to create another shared report "Product Performance Report" that parses the same sourcetype differently as roughly 30% of the entries in product log contain performance data that we would like to chart. This extraction pulls out the 'duration' and 'url' fields from those entries.

^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s\-\-Done\s\[(?P<dur>.*)\]\s\[(?P<url>.*)\].*$

How can I apply the 2nd extraction 'rule' to the same sourcetype but only use it when viewing the "Performance Report"? Is there a better approach to get the same results?

Sample Entries:

2015-01-23 00:02:06,161 INFO   [ 68] :-> foo bar
2015-01-23 00:02:26,177 INFO   [ 65] :-> --Done [   15.581] [http://the.url.org/mickey/mouse]
2015-01-23 00:02:36,302 INFO   [ 65] :-> bla bla bla
2015-01-23 00:02:36,349 INFO   [ 65] :-> --Done [  203.111] [http://the.url.org/donald/duck]

Re: How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

somesoni2 — Fri, 23 Jan 2015 18:53:29 GMT

The field extraction is done at sourcetype level so I am not sure if you can conditionally choose which field extractions to use. What I would suggest is the define two field extractiosn stanza, for your sourcetype.

Props.conf in Search Head
[product]
EXTRACT-general = ^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s(?P<body>.*)$
EXTRACT-perf = :->\s\-\-Done\s\[(?P<dur>.*)\]\s\[(?P<url>.*)\].*$

This should create fields ts, level,tid, body for all events. It will also create dur and url for all events but for non performance data, they would be null. So, for General report your just refer fields ts, level,tid, body and for Performance report, just use fields ts, level,tid, dur, url.

Re: How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

tlbj6142 — Fri, 23 Jan 2015 19:10:23 GMT

Thanks. That makes some sense. I'll give that a try. Can I do that through the admin UI? My operation's staff doesn't give me direct access to props.conf.

Re: How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

somesoni2 — Fri, 23 Jan 2015 19:13:13 GMT

Yes, You can add field extraction through Splunk Web's admin pages.
http://docs.splunk.com/Documentation/Splunk/6.2.1/Knowledge/Managesearch-timefieldextractions

Re: How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

tlbj6142 — Fri, 23 Jan 2015 19:34:28 GMT

It worked. Thanks.

Re: How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

tlbj6142 — Fri, 23 Jan 2015 19:35:51 GMT

Make your comment an 'answer' so I can mark the question as answered. Thanks.

Re: How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

somesoni2 — Fri, 23 Jan 2015 19:41:14 GMT

Glad it helped. Here you go.