Best Practice - Multiple source types

ski98033 — Thu, 07 Jun 2012 21:45:00 GMT

Hi,

What is the best practice when looking at syslogs:

Split the syslog into multiple files, one for each source type (e.g. mail, syslog, apache, etc.) and then apply one source type to each file.
Read in the syslog and then have the indexer apply the source type based upon entries in the transforms and props files.

What are the pluses and minutes of each option.

cheers,

ski

Re: Best Practice - Multiple source types

kristian_kolb — Fri, 08 Jun 2012 07:09:39 GMT

Hi,

I would say that if you can be bothered to configure your syslog-server to split incoming data into separate files you'd be a lot better off. Since the syslog daemon is aware of the originating IP/hostname as well as facility, you can build a directory structure like /var/log/my_syslog/<hostname>/<ip-address>.log or something similar.

This is more certain than relying on Splunk being able to parse out the hostname from each event, which is default splunk behaviour for the syslog sourcetype. This will work in almost all cases, but I've seen network appliances that log without a space after the hostname, resulting in hosts like

switch01snmpUtil
switch01snmpMgmt
switch01somethingElse

Thus you'll get a new host for each type of message from a single device, which is not so good, and unfortunately hard/impossible to correct on the splunk server.

Hope this helps,

Kristian

topic Best Practice - Multiple source types in Getting Data In

Best Practice - Multiple source types

Re: Best Practice - Multiple source types