Splunk Search

How to define specific characters within angle brackets in my syslog data as certain values on my heavy forwarder or in a search?

jcrosby21
Path Finder

I have syslog information being sent to my heavy forwarder and I'd like to define a specific translation for one piece of information. The number in the <> brackets translates to Error, Warning, Info, Debug, etc.

Jun 28 13:18:14 xxx.xxx.xxx.xxx Jun 28 13:16:44 vThunder a10logd: [SYSTEM]<6> Running co
Jun 28 13:19:00 xxx.xxx.xxx.xxx Jun 28 13:17:31 vThunder a10logd: [SYSTEM]<4> Local auth
Jun 28 13:19:00 xxx.xxx.xxx.xxx Jun 28 13:17:31 vThunder a10logd: [SYSTEM]<5> A web sess
Jun 28 13:19:20 xxx.xxx.xxx.xxx Jun 28 13:17:50 vThunder a10logd: [CFGMGR]<7> Doesn't fi
Jun 28 13:19:20 xxx.xxx.xxx.xxx Jun 28 13:17:50 vThunder a10logd: [VCS]<6> dcs config se
Jun 28 13:19:20 xxx.xxx.xxx.xxx Jun 28 13:17:50 vThunder a10logd: [VCS]<6> dcs config se
Jun 28 13:22:15 xxx.xxx.xxx.xxx Jun 28 13:20:46 vThunder a10logd: [SYSTEM]<5> Session ID
Jun 28 13:22:15 xxx.xxx.xxx.xxx Jun 28 13:20:46 vThunder a10logd: [SYSTEM]<6> Session ti
Jun 28 13:24:09 xxx.xxx.xxx.xxx Jun 28 13:22:39 vThunder a10logd: [SYSTEM]<4> Local auth
Jun 28 13:24:09 xxx.xxx.xxx.xxx Jun 28 13:22:39 vThunder a10logd: [SYSTEM]<5> A web sess

So:
7=Debug
6=Info
5=Warning
4=Error

However, in my searching, I'm not sure the right way to accomplish this.

What I would like in the search is to be able to filter to just the warnings (5), but do it with the word "warning" instead of remembering that number 5 is the warning level.

Is that a new index-time field? Can I just add the field as a lookup to my sourcetype?

0 Karma
1 Solution

woodcock
Esteemed Legend

You do this by configuring an automatic lookup:

https://docs.splunk.com/Documentation/Splunk/6.4.1/Knowledge/Makeyourlookupautomatic

Then you can specify your search by the new field with the name ( newfield="Warning" ) instead of the old field by the number ( oldfield=5 ). This assumes that you have already created a configuration to create the oldfield.

View solution in original post

woodcock
Esteemed Legend

You do this by configuring an automatic lookup:

https://docs.splunk.com/Documentation/Splunk/6.4.1/Knowledge/Makeyourlookupautomatic

Then you can specify your search by the new field with the name ( newfield="Warning" ) instead of the old field by the number ( oldfield=5 ). This assumes that you have already created a configuration to create the oldfield.

jcrosby21
Path Finder

So I've broken out some new indexed fields via props/transforms/fields on my HF, but you're thinking a search-time automatic lookup for the additional field rather than some additional transformation on my HF?

0 Karma

woodcock
Esteemed Legend

Yes, exactly. It is easier to maintain (update when new values occur) and you still get all the benefits of it being indexed because the oldfield is indexed and your search will automatically convert your specified newfield="warning" to oldfield=5 when your search is sent to the indexers.

0 Karma

jcrosby21
Path Finder

Hmm, so I had trouble putting my CSV in $SPLUNK_HOME/etc/system/lookups and my stanza in transforms.conf $SPLUNK_HOME/etc/system/local on my search head - it kept not finding my CSV for some reason - but when I moved them both under the search app I got it working in the query and was able to add a sourcetype stanza to my props.conf in the 'search' app and got the auto-lookup working.

Looks pretty good. Thanks!

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...