Splunk Search
Highlighted

How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

Explorer

Most of the time we use a shared report ("General Product Report") to view our logs for sourcetype="product". I created a field extraction rule to parse each entry into 7-8 fields (the sample below has been trimmed down for brevity).

^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s(?P<body>.*)$

I'd like to create another shared report "Product Performance Report" that parses the same sourcetype differently as roughly 30% of the entries in product log contain performance data that we would like to chart. This extraction pulls out the 'duration' and 'url' fields from those entries.

^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s\-\-Done\s\[(?P<dur>.*)\]\s\[(?P<url>.*)\].*$

How can I apply the 2nd extraction 'rule' to the same sourcetype but only use it when viewing the "Performance Report"? Is there a better approach to get the same results?

Sample Entries:

2015-01-23 00:02:06,161 INFO   [ 68] 😆 foo bar
2015-01-23 00:02:26,177 INFO   [ 65] 😆 --Done [   15.581] [http://the.url.org/mickey/mouse]
2015-01-23 00:02:36,302 INFO   [ 65] 😆 bla bla bla
2015-01-23 00:02:36,349 INFO   [ 65] 😆 --Done [  203.111] [http://the.url.org/donald/duck]
Highlighted

Re: How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

SplunkTrust
SplunkTrust

The field extraction is done at sourcetype level so I am not sure if you can conditionally choose which field extractions to use. What I would suggest is the define two field extractiosn stanza, for your sourcetype.

Props.conf in Search Head
[product]
EXTRACT-general = ^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s(?P<body>.*)$
EXTRACT-perf = :->\s\-\-Done\s\[(?P<dur>.*)\]\s\[(?P<url>.*)\].*$

This should create fields ts, level,tid, body for all events. It will also create dur and url for all events but for non performance data, they would be null. So, for General report your just refer fields ts, level,tid, body and for Performance report, just use fields ts, level,tid, dur, url.

View solution in original post

Highlighted

Re: How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

Explorer

Thanks. That makes some sense. I'll give that a try. Can I do that through the admin UI? My operation's staff doesn't give me direct access to props.conf.

0 Karma
Highlighted

Re: How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

SplunkTrust
SplunkTrust

Yes, You can add field extraction through Splunk Web's admin pages.
http://docs.splunk.com/Documentation/Splunk/6.2.1/Knowledge/Managesearch-timefieldextractions

Highlighted

Re: How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

Explorer

It worked. Thanks.

0 Karma
Highlighted

Re: How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

Explorer

Make your comment an 'answer' so I can mark the question as answered. Thanks.

0 Karma
Highlighted

Re: How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

SplunkTrust
SplunkTrust

Glad it helped. Here you go.

0 Karma