Trouble getting syslog_ng to work on a standalone Splunk instance

[Skins]

Ive install syslog-ng on a standalone splunk instance but cannot get it running - ive looked at the following guide :

https://www.splunk.com/blog/2016/03/11/using-syslog-ng-with-splunk.html

using a syslog gen i can send a message directly to splunk as a direct input, but then i disable that and configure syslog-ng. the service starts and is listening but nothing is written to a file

[root@centos-6-1 syslog-ng]# netstat -anp | grep 514
udp 0 0 0.0.0.0:514 0.0.0.0:* 13833/syslog-ng

sending a facility 7 syslog message from cmd line:

SyslogGen.exe -t:x.x.x.x -f:7 -s:7 -h:myhost -m:"Too many bytes.\x0D\x0A"

@version:3.2

syslog-ng configuration file.

This should behave pretty much like the original syslog on RedHat. But

it could be configured a lot smarter.

See syslog-ng(8) and syslog-ng.conf(5) for more information.

options {
flush_lines (0);
time_reopen (10);
log_fifo_size (1000);
long_hostnames (off);
use_dns (no);
use_fqdn (no);
create_dirs (no);
keep_hostname (yes);
};
source s_sys {
udp(port(514));
};

destination d_cons { file("/dev/console"); };

destination d_mesg { file("/opt/syslog-ng/$HOST/$YEAR-$MONTH-$DAY-test.log"); };

destination d_auth { file("/var/log/secure"); };

destination d_mail { file("/var/log/maillog" flush_lines(10)); };

destination d_spol { file("/var/log/spooler"); };

destination d_boot { file("/opt/syslog-ng/$HOST/$YEAR-$MONTH-$DAY-test1.log"); };

destination d_cron { file("/var/log/cron"); };

destination d_kern { file("/var/log/kern"); };

destination d_mlal { usertty("*"); };

filter f_kernel { facility(kern); };

filter f_default { level(info..emerg) and
not (facility(mail)
or facility(authpriv)
or facility(cron)); };

filter f_auth { facility(authpriv); };

filter f_mail { facility(mail); };

filter f_emergency { level(emerg); };

filter f_boot { facility(local7); };

filter f_cron { facility(cron); };

log { source(s_sys); filter(f_kernel); destination(d_cons); };

log { source(s_sys); filter(f_kernel); destination(d_kern); };

log { source(s_sys); filter(f_default); destination(d_mesg); };

log { source(s_sys); filter(f_auth); destination(d_auth); };

log { source(s_sys); filter(f_mail); destination(d_mail); };

log { source(s_sys); filter(f_emergency); destination(d_mlal); };

log { source(s_sys); filter(f_news); destination(d_spol); };

log { source(s_sys); filter(f_boot); destination(d_boot); };

log { source(s_sys); filter(f_cron); destination(d_cron); };

vim:ft=syslog-ng:ai:si:ts=4:sw=4:et:

gratzi

[frobert]

Hi, in your syslog-ng configuration, change create_dirs (no); to yes, otherwise syslog-ng will use only existing directories.
If that doesn't help, check your SELinux settings and adjust them if needed to permit syslog-ng to write in the directories you want to.

[starcher]

You are better off using syslog-ng to write to files and use a Universal Forward to pick up those files. Never send syslog straight to Splunk via the network. You will cause yourself all sorts of tears in different source types, dropping data when restarting Splunk etc.

[tmarlette]

you don't even need syslog-ng if you're using Splunk, just install a heavy forwarder and listen on the relevant ports.

To organize your data feeds, just use props.conf and transforms.conf to send data to the approriate indexes and sourcetypes.

Here's an example from my environment, and we're retiring syslog and replacing it with Splunk.

inputs.conf

[udp://514]
connection_host = dns
index = network_syslog
sourcetype = syslog_pool
disabled = 0

transforms.conf

[prime_index]
SOURCE_KEY = MetaData:Host
REGEX = (myhost05\.|myFireWall02|10.10.10.126)
FORMAT = network
DEST_KEY = _MetaData:Index

[prime_sourcetype]
SOURCE_KEY = MetaData:Host
REGEX = (myhost05\.|myFireWall02|10.10.10.126)
FORMAT = sourcetype::wips_events
DEST_KEY = MetaData:Sourcetype

props.conf

[source::udp:514]
TRANSFORMS-syslog_handling = prime_index,prime_sourcetype

With these settings, my splunk HF is listening on UDP port 514, and any data that matches the hostname of 'myhost05, myFireWall02 and 10.10.10.126 are all being sent to the network index with the sourcetype wips_events

You may need to open UDP 514 in iptables, but it looks like that might already be done.