Getting Data In

Best Practice - Multiple source types

ski98033
Explorer

Hi,

What is the best practice when looking at syslogs:

  1. Split the syslog into multiple files, one for each source type (e.g. mail, syslog, apache, etc.) and then apply one source type to each file.
  2. Read in the syslog and then have the indexer apply the source type based upon entries in the transforms and props files.

What are the pluses and minutes of each option.

cheers,

ski

Tags (1)
0 Karma

kristian_kolb
Ultra Champion

Hi,

I would say that if you can be bothered to configure your syslog-server to split incoming data into separate files you'd be a lot better off. Since the syslog daemon is aware of the originating IP/hostname as well as facility, you can build a directory structure like /var/log/my_syslog/<hostname>/<ip-address>.log or something similar.

This is more certain than relying on Splunk being able to parse out the hostname from each event, which is default splunk behaviour for the syslog sourcetype. This will work in almost all cases, but I've seen network appliances that log without a space after the hostname, resulting in hosts like

switch01snmpUtil
switch01snmpMgmt
switch01somethingElse

Thus you'll get a new host for each type of message from a single device, which is not so good, and unfortunately hard/impossible to correct on the splunk server.

Hope this helps,

Kristian

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...