Splunk Search

How to extract 2 different sets of fields for the same sourcetype, but only use each set when viewed in 2 separate reports?

tlbj6142
Explorer

Most of the time we use a shared report ("General Product Report") to view our logs for sourcetype="product". I created a field extraction rule to parse each entry into 7-8 fields (the sample below has been trimmed down for brevity).

^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s(?P<body>.*)$

I'd like to create another shared report "Product Performance Report" that parses the same sourcetype differently as roughly 30% of the entries in product log contain performance data that we would like to chart. This extraction pulls out the 'duration' and 'url' fields from those entries.

^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s\-\-Done\s\[(?P<dur>.*)\]\s\[(?P<url>.*)\].*$

How can I apply the 2nd extraction 'rule' to the same sourcetype but only use it when viewing the "Performance Report"? Is there a better approach to get the same results?

Sample Entries:

2015-01-23 00:02:06,161 INFO   [ 68] 😆 foo bar
2015-01-23 00:02:26,177 INFO   [ 65] 😆 --Done [   15.581] [http://the.url.org/mickey/mouse]
2015-01-23 00:02:36,302 INFO   [ 65] 😆 bla bla bla
2015-01-23 00:02:36,349 INFO   [ 65] 😆 --Done [  203.111] [http://the.url.org/donald/duck]
1 Solution

somesoni2
SplunkTrust
SplunkTrust

The field extraction is done at sourcetype level so I am not sure if you can conditionally choose which field extractions to use. What I would suggest is the define two field extractiosn stanza, for your sourcetype.

Props.conf in Search Head
[product]
EXTRACT-general = ^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s(?P<body>.*)$
EXTRACT-perf = :->\s\-\-Done\s\[(?P<dur>.*)\]\s\[(?P<url>.*)\].*$

This should create fields ts, level,tid, body for all events. It will also create dur and url for all events but for non performance data, they would be null. So, for General report your just refer fields ts, level,tid, body and for Performance report, just use fields ts, level,tid, dur, url.

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

The field extraction is done at sourcetype level so I am not sure if you can conditionally choose which field extractions to use. What I would suggest is the define two field extractiosn stanza, for your sourcetype.

Props.conf in Search Head
[product]
EXTRACT-general = ^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s(?P<body>.*)$
EXTRACT-perf = :->\s\-\-Done\s\[(?P<dur>.*)\]\s\[(?P<url>.*)\].*$

This should create fields ts, level,tid, body for all events. It will also create dur and url for all events but for non performance data, they would be null. So, for General report your just refer fields ts, level,tid, body and for Performance report, just use fields ts, level,tid, dur, url.

tlbj6142
Explorer

Thanks. That makes some sense. I'll give that a try. Can I do that through the admin UI? My operation's staff doesn't give me direct access to props.conf.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Yes, You can add field extraction through Splunk Web's admin pages.
http://docs.splunk.com/Documentation/Splunk/6.2.1/Knowledge/Managesearch-timefieldextractions

tlbj6142
Explorer

Make your comment an 'answer' so I can mark the question as answered. Thanks.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Glad it helped. Here you go.

0 Karma

tlbj6142
Explorer

It worked. Thanks.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...