Our splunk server is also our syslog-ng server, so Splunk uses the local /var/log/messages which contains entries from all of our hosts. Each log message has facility and level by adding this to syslog-ng.conf on the syslog server:
destination messages {
file("/var/log/messages" template("$DATE $FULLHOST $MESSAGE [facility=$FACILITY level=$LEVEL]\n") ); };
log { source(src); filter(f_messages); destination(messages); };
}
Each client pushes to the syslog-ng server via tcp(514). When syslog gets messages from tcp 514, it puts it in the messages file in that format spefified by template(). Splunk automatically creates fields from anything with "field=foo", so you'll get fields for facility and level.
Again, Splunk is not listening on 514, but only indexing the local /var/log/messages. Syslog-ng is the one doing the listening.
... View more