handling syslog...

[a212830]

Hi,

We are in the midst of implementing Splunk to handle syslog from all of our network devices. I've configured rsyslog to write the logs to a YYYY/MM/DD directory, in a "system-hostname.log" format.
Does anyone have a suggestion on how to handle all of these formats (a dozen+), and allow engineering to add new device types, without intervention on the Splunk side?

[kristian_kolb]

I would re-engineer the rsyslog configuration so that you have a directory structure like

/var/log/netscreen/<hostname>/yyyy-mm-dd.log
/var/log/f5/<hostname>/yyyy-mm-dd.log

etc. You'd have to set up the device-type subdirs manually and make sure that logs from a certain IP gets written to the correct dir. I believe that you can use the rsyslog variables %HOSTNAME% or $fromhostip to create the hostname directories automatically, but you'll have to maintain the mapping of hostname (or IP) to device-type in the rsyslog conf. However, this is probably fairly static, and can be set up regardless of whether the devices are actually sending any logs (yet).

Then you can have a fairly 'static' Splunk config like so;

[monitor:///var/log/netscreen]
host_segment = 4
index = your_index
sourcetype = netscreen_syslog
ignoreOlderThan = 7d

[monitor:///var/log/f5]
host_segment = 4
index = your_index
sourcetype = f5_syslog
ignoreOlderThan = 7d

etc.

Hope this helps,

Kristian

[a212830]

Currently, the props files has entries for each device model - netscreens, checkpoint, f5.... I'd like to come up with a way to let engineering route the messages to splunk, and not be required (at least not initially) to add some code to process the files. We don't use a generic syslog sourcetype, as it's not detailed enough. So, we have a f5_syslog, netscreen_syslog... which actually comes in handy, since there are a number of extracts that only pertain to certain models.

[lguinn2]

What do you mean by "without intervention on the Splunk side"? Also, what are the differences in the formats? syslog generally defines a generic format which Splunk understands; if all of the logs follow this format, then the syslog sourcetype should work fine.