Change host name from syslog

[pil321]

I'm saving syslog on a server and forwarding to Splunk. There is one rogue host that saves the log as this:

2017-08-20T09:31:29.383904-04:00 2017-08-20 09: 31:29 switch2B76AE NOTE SNTP[SNTP]

Since there are 2 time stamps, Splunk sets the host name as the date (2017-08-20). All other hosts with the 'syslog' sourcetype are behaving properly.

I think what I need to do is manipulate props.conf and transforms.conf, but wouldn't that also change every host that uses the 'syslog' sourcetype?

[cpetterborg · Accepted solution]

You can use a regular expression for the name which would either start with a alphabetic character or match an IP address to set the hostname properly. This method would allow you to use the same host extraction across all syslog data, and be able to have any number of hosts send their syslog data in without having to worry about changing the list of those to use the specific host extraction. Here is a regular expression that should be able to handle the host extraction for you:

^[-\s\d:.T]+?\s(?P<host>([a-zA-Z][\w\.-]*|\d+\.\d+\.\d+\.\d+))

This regular expression works on the following data set:

2017-08-20T09:31:29.383904-04:00 2017-08-20 09: 31:29 switch2B76AE NOTE SNTP[SNTP]
2017-08-20T09:31:29.383904-04:00 2017-08-20 09: 31:29 10.120.130.140 NOTE SNTP[SNTP]
08-20T09:31:29.383904-04:00 10.120.130.140 NOTE SNTP[SNTP]
2017-08-20 09: 31:29 switch-2B7_6AE.mydomain.com NOTE SNTP[SNTP]

The only place that I see this failing is if there is an alphabetic month in the date (like Aug), but I don't think that there are any syslog date formats that do that.

I would suggest doing this in the props.conf and transforms.conf. If you want more explicit information about doing this, comment here and I'll supply more specific information about that.

[pil321]

Thank you for your response cpetterborg. Could you please provide an example on how the props.conf and transforms.conf would need to be for this to work?

[cpetterborg]

Here are example file contents for extracting the host as I described previously. I haven't tried these in this specific instance, but I believe it should work without much modification. You will have to change the sourcetype ( mysyslog ) in the props.conf to match your sourcetype. These should be on the indexers, not on forwarders or search heads.

transforms.conf:

[hostextract]
REGEX =  ^[-\s\d:.T]+?\s(([a-zA-Z][\w\.-]*|\d+\.\d+\.\d+\.\d+))
SOURCE_KEY = _raw
DEST_KEY = MetaData:Host
FORMAT = host::$1

props.conf:

[mysyslog]
TRANSFORMS-hostextract = hostextract

[pil321]

That did the trick. Thank you kindly!

[jkat54]

On syslog you can write a filter that puts this host in a specific file and then read that file in with its own sourcetype, hostname, etc.