topic Re: System Name from Syslog File in Getting Data In

System Name from Syslog File

dbritch — Fri, 29 Oct 2010 13:12:17 GMT

I'm using splunk to monitor /var/log on a RHEL-5.5 syslog server. It's running rsyslog, not syslog-ng. For some log messages, Splunk can get the name of the originating node, but on for others it simply attributes them to the log server.

How can I get Splunk to use the node name in all cases?

Thanks,

David

Re: System Name from Syslog File

ftk — Fri, 29 Oct 2010 22:31:32 GMT

Please post some example lines from your the syslog logs as well as the directory structure used to store them, so we can determine how to best extract a host value. Thanks

Re: System Name from Syslog File

dbritch — Sat, 30 Oct 2010 02:28:58 GMT

Sure - here's an example. Everything from /var/log/secure is attributed to r00n06, my syslog and splunk server:

2010-10-29T19:27:29+00:00 r07n15 sshd[8158]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r07n15 sshd[8160]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r01n40 sshd[24669]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r01n40 sshd[24671]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r07n11 sshd[9969]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r07n11 sshd[9971]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r03n03 sshd[13527]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r03n03 sshd[13526]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r02n07 sshd[21721]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r02n07 sshd[21722]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r06n26 sshd[30827]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r06n26 sshd[30829]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r10n32 sshd[3410]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r10n32 sshd[3411]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r09n27 sshd[637]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r09n27 sshd[639]: Connection closed by 10.0.0.6
2010-10-29T19:27:30+00:00 r05n06 sshd[7678]: Connection closed by 10.253.0.6
2010-10-29T19:27:30+00:00 r05n06 sshd[7680]: Connection closed by 10.0.0.6

Re: System Name from Syslog File

dbritch — Sat, 30 Oct 2010 03:05:05 GMT

I gess that should have been a comment rather than an answer - sorry. I'm just learning this interface.

dbr

Re: System Name from Syslog File

southeringtonp — Thu, 11 Nov 2010 09:25:29 GMT

As you have discovered, the default hostname extraction for syslog does not work with rsyslog's datestamp format. You will need to add your own.

In transforms.conf:

[rsyslog-host]
DEST_KEY = MetaData:Host
REGEX = ^[\d\-T\+:]+ (\S+)
FORMAT = host::$1

In props.conf (replace rsyslog below with your sourcetype name as needed):

[rsyslog]
TRANSFORMS-host = rsyslog-host

Re: System Name from Syslog File

wwwdrich — Tue, 25 Jan 2011 02:27:23 GMT

I couldn't get the above working properly with 4.1.6, so I ended up using a slightly modified transforms.conf file:

[rsyslog-native-host]
DEST_KEY = MetaData:Host
REGEX = [\d\-\+:.]+T[\d\-\+:.]+\s+(\S+)
FORMAT = host::$1

Another option is to use the RSYSLOG_TraditionalForwardFormat template in your rsyslog.conf file. That will forward messages that look like standard syslog messages.

The problem that neither of these solves however is that splunk is adding the timestamp and host it finds in the message header to the message. So even with a proper regex to extract the hostname, you still end up with messages like this in your logs:

Jan 24 10:09:07 localhost 2011-01-24T10:09:07.974181-08:00 xxxx snmpd[29862]: Received SNMP packet(s) from UDP: [127.0.0.1]:47553

Does anyone know of a way to strip that off of the logged messages w/o jumping through too many hoops?

Re: System Name from Syslog File

wwwdrich — Tue, 25 Jan 2011 03:06:40 GMT

I played a bit more -- if you use the RSYSLOG_TraditionalForwardFormat over TCP instead of UDP, you can use the syslog sourcetype with Splunk an it works just fine.

The only problem is that it will have a "<##>" at the beginning of every message. That is the priority coming from the syslog message.

Re: System Name from Syslog File

enno_davids — Tue, 23 Sep 2014 00:42:52 GMT

Is it something with my browser or are the \'s (slashes) missing from the regex's above?