topic Re: How do I extract syslog code to carry out a lookup and create new indexed fields for syslog facility and severity? in Getting Data In

How do I extract syslog code to carry out a lookup and create new indexed fields for syslog facility and severity?

corners — Thu, 17 Mar 2016 10:36:05 GMT

I'm running Splunk 5.0.4. In the environment I have 2 servers
deploy/heavy forwarder
Search head/indexer.

On the heavy forwarder I have setup the listener for syslog udp:514

C:\Program Files\Splunk\etc\system\local\inputs.conf
[udp://514]
connection_host = dns
sourcetype = syslog
no_priority_stripping = true
no_appending_timestamp = true

This is working and forwarding on the syslog events to the indexer and the syslog code is being included but the timestamp is still being included also eg -

<189>: 2016 Mar 17 09:08:18.325 gmt

What I want to do is use the syslog code <189> to do a lookup against a csv to find the matching facility and severity and have the 2 values included as indexed fields within Splunk indexer.

I have atempted to do this by including the following on the Heavy Forwarder (Is this the correct place and server??)

C:\Program Files\Splunk\etc\apps\Aggregator_f\local\props.conf
[source::udp:514]
EXTRACT-extractSyslogcode = (?i)^<(?P<syslog_code>[^>]+)
LOOKUP-SyslogCode = syslog_facility_severity_codes code AS syslog_code OUTPUTNEW facility AS sys_facility, severity AS sys_severity

C:\Program Files\Splunk\etc\apps\Aggregator_f\local\transforms.conf
[syslog_facility_severity_codes]
filename = syslog-codes.csv

The csv file for the lookup is located in

C:\Program Files\Splunk\etc\apps\Health_Aggregator_f\lookups\syslog-codes.csv

If anyone can provide any assistance with this it would be greatly received.

thanks
Steve

Re: How do I extract syslog code to carry out a lookup and create new indexed fields for syslog facility and severity?

lguinn2 — Thu, 17 Mar 2016 17:59:39 GMT

"What I want to do is use the syslog code to do a lookup against a csv to find the matching facility and severity and have the 2 values included as indexed fields within Splunk indexer."

Lookups only happen at search time. Therefore, you can't put the lookup configuration on a forwarder (well, you can, but it doesn't do anything). The lookup should be configured on the search head (or the indexer if you don't have a separate search head). This also means that you can't index the result of a lookup.
In general (which means 99.99% of the time), index-time fields are a bad idea in Splunk. I know that this is completely counter-intuitive, but it is true. Make your lookup automatic and it will provide the additional fields at search time; to the user, it will appear that these fields are indexed.

Re: How do I extract syslog code to carry out a lookup and create new indexed fields for syslog facility and severity?

corners — Fri, 18 Mar 2016 09:27:48 GMT

Thanks for replying.
I did move the settings over to the indexer yesterday after further reading around lookups.
This has resulted in the additional fields being generated at search time.