System Name from Syslog File

[dbritch]

I'm using splunk to monitor /var/log on a RHEL-5.5 syslog server. It's running rsyslog, not syslog-ng. For some log messages, Splunk can get the name of the originating node, but on for others it simply attributes them to the log server.

How can I get Splunk to use the node name in all cases?

Thanks,

David

[enno_davids]

Is it something with my browser or are the \'s (slashes) missing from the regex's above?

[wwwdrich]

I couldn't get the above working properly with 4.1.6, so I ended up using a slightly modified transforms.conf file:

[rsyslog-native-host]
DEST_KEY = MetaData:Host
REGEX = [\d\-\+:.]+T[\d\-\+:.]+\s+(\S+)
FORMAT = host::$1

Another option is to use the RSYSLOG_TraditionalForwardFormat template in your rsyslog.conf file. That will forward messages that look like standard syslog messages.

The problem that neither of these solves however is that splunk is adding the timestamp and host it finds in the message header to the message. So even with a proper regex to extract the hostname, you still end up with messages like this in your logs:

Jan 24 10:09:07 localhost 2011-01-24T10:09:07.974181-08:00 xxxx snmpd[29862]: Received SNMP packet(s) from UDP: [127.0.0.1]:47553

Does anyone know of a way to strip that off of the logged messages w/o jumping through too many hoops?

[wwwdrich]

I played a bit more -- if you use the RSYSLOG_TraditionalForwardFormat over TCP instead of UDP, you can use the syslog sourcetype with Splunk an it works just fine.

The only problem is that it will have a "<##>" at the beginning of every message. That is the priority coming from the syslog message.

[southeringtonp]

As you have discovered, the default hostname extraction for syslog does not work with rsyslog's datestamp format. You will need to add your own.

In transforms.conf:

[rsyslog-host]
DEST_KEY = MetaData:Host
REGEX = ^[\d\-T\+:]+ (\S+)
FORMAT = host::$1

In props.conf (replace rsyslog below with your sourcetype name as needed):

[rsyslog]
TRANSFORMS-host = rsyslog-host

[dbritch]

Sure - here's an example. Everything from /var/log/secure is attributed to r00n06, my syslog and splunk server:

2010-10-29T19:27:29+00:00 r07n15 sshd[8158]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r07n15 sshd[8160]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r01n40 sshd[24669]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r01n40 sshd[24671]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r07n11 sshd[9969]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r07n11 sshd[9971]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r03n03 sshd[13527]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r03n03 sshd[13526]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r02n07 sshd[21721]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r02n07 sshd[21722]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r06n26 sshd[30827]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r06n26 sshd[30829]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r10n32 sshd[3410]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r10n32 sshd[3411]: Connection closed by 10.0.0.6
2010-10-29T19:27:29+00:00 r09n27 sshd[637]: Connection closed by 10.253.0.6
2010-10-29T19:27:29+00:00 r09n27 sshd[639]: Connection closed by 10.0.0.6
2010-10-29T19:27:30+00:00 r05n06 sshd[7678]: Connection closed by 10.253.0.6
2010-10-29T19:27:30+00:00 r05n06 sshd[7680]: Connection closed by 10.0.0.6

[dbritch]

I gess that should have been a comment rather than an answer - sorry. I'm just learning this interface.

dbr

[ftk]

Please post some example lines from your the syslog logs as well as the directory structure used to store them, so we can determine how to best extract a host value. Thanks