Getting Data In

Automatic Sourcetype Tagging on Syslog Server

Path Finder

Hi everyone,

In my environment, we are collecting logs from several types of devices on a syslog server, then forwarding the data to the indexers. Most of the data comes into the syslog server on port 514. We have brocade, juniper, f5, cisco, bluecoat, and some others. This example I found to use syslog-ng with Splunk I think is fantastic:

Essentially, the configuration takes the incoming data and automatically separates the data by device type. So the syslog server would put juniper into /logs/juniper, brocade into /logs/brocade, f5 into /logs/f5, etc. Then I could just create an input stanza in Splunk that says anything /juniper tag with the juniper sourcetype, /brocade gets brocade sourcetype, etc.

This is exactly what we would want. My question though, which comes from a lack of understanding syslog in depth, is how would we take all of this different data coming into syslog on port 514 and have syslog automatically determine what type of device the data is coming from? In the example, it looks like that might be what the filters are doing... but if that's true, how would I figure out a filter to identify each device type's data?

Oh and I should add that each device type is using the same standard syslog format I believe, and when looking at the different data, there didn't seem to be any key indicators that identifies what type of device the data is coming from.

Path Finder

Thank you all for replying with suggestions, my team really appreciates the input! We discussed all of the options. We are very much leaning towards @woodcock's suggestion to use a different port for each sourcetype, and for the devices that can only send on port 514, using a naming convention so that we can specificially apply the sourcetype to those devices. We felt that this would be the most simple, straightforward, and viable option in our environment.

Once we finalize a decision and configure it in our environment, I'll report back here with how it goes, so that it can help anyone else that reads this or runs into a similar situation.


0 Karma

Esteemed Legend

Be sure to click Accept and UpVote where appropriate when you come back.

0 Karma

Ultra Champion

Or run multiple instances of rsyslog/syslog-ng on separate (virtual) IPs and segregate the traffic that way, if you’re stuck with using port 514.

Esteemed Legend

BRILLIANT! I don't know why I never thought of that!!!!

0 Karma

Esteemed Legend

The best way to build a future-proof syslog server is to put each sourcetype on a different port. Most syslog export functions on most applications do allow this. There are a very few big name products that cannot. As much as possible, use this approach and fight back hard against the security ninnies who will complain; there is nothing at all insecure about using more ports. When you cannot, make sure that the hostnames of the servers encode their function; for example, your DCs use a hostnaming convention like location-DC-instance (e.g. DAL-DC-01). Then you can build a hostname filter ( *-DC-* ) to split it out.

Super Champion

Hi @R_B,

may be your syslog-ng filters are not working properly.

syslog-ng requires to configure source ,filter and destination.

example filter from the document is below
filter f_cisco_asa { match(“%ASA” value(“PROGRAM”)) or match(“%ASA” value(“MESSAGE”)); };

This looks for %ASA in all UDP 514 messages. if it matches then will write to destination
destination d_cisco_asa { file(“/home/syslog/logs/cisco/asa/$HOST/$YEAR-$MONTH-$DAY-cisco-asa.log” create_dirs(yes)); };
using below line.
log { source(s_network); filter(f_cisco_asa); destination(d_cisco_asa); };

If this helps, give a like below.


Can you not use source field to differentiate the data by device?

Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...