topic Re: syslog-ng to rsyslog in Getting Data In

syslog-ng to rsyslog

dmed — Sat, 27 Nov 2021 19:07:02 GMT

Hi dear splunk community,

Can someone help me to convert/translate the following syslog-ng config to the corresponding rsyslog server side config please ?

The standard syslog-ng.conf file simply includes the statements below which are in a file in the conf.d dir like so:

@include "/etc/syslog-ng/conf.d/*.conf"

I'd really appreciate it. It doesn't have to be perfect or exact or even completely converted, as long as most of it can be translated...the main concerns being the audit logs and all the rest of the program logs...

Thanks so very much,

source s_remote { syslog(port(514), transport(tcp), flags(), max-connections(100),log-fetch-limit(100),log_iw_size(20000)); };

destination d_kern { file("/var/log/syslog-to-splunk/$HOST/kernel.log" create-dirs(yes)); };
destination d_mail { file("/var/log/syslog-to-splunk/$HOST/mail.log" create-dirs(yes)); };
destination d_daemon { file("/var/log/syslog-to-splunk/$HOST/daemon.log" create-dirs(yes)); };
destination d_auth { file("/var/log/syslog-to-splunk/$HOST/auth.log" create-dirs(yes)); };
destination d_cron { file("/var/log/syslog-to-splunk/$HOST/cron.log" create-dirs(yes)); };
destination d_security { file("/var/log/syslog-to-splunk/$HOST/audit.log" create-dirs(yes)); };
# All else.
destination d_rest { file("/var/log/syslog-to-splunk/$HOST/program/$PROGRAM.log" create-dirs(yes)); };

filter f_kern { facility(kern); };
filter f_mail { facility(mail); };
filter f_daemon { facility(daemon, user, syslog); };
filter f_auth { facility(auth, authpriv, security); };
filter f_cron { facility(cron); };
filter f_security { facility(kern, auth, authpriv, security, local7); };
filter f_rest { not facility(auth, authpriv, cron, kern, mail, user, security, syslog); };

log { source(s_remote); filter(f_kern); destination(d_kern); };
log { source(s_remote); filter(f_mail); destination(d_mail); };
log { source(s_remote); filter(f_daemon); destination(d_daemon); };
log { source(s_remote); filter(f_auth); destination(d_auth); };
log { source(s_remote); filter(f_cron); destination(d_cron); };
log { source(s_remote); filter(f_security); destination(d_security); };
log { source(s_remote); filter(f_rest); destination(d_rest); };

Re: syslog-ng to rsyslog

PickleRick — Sun, 28 Nov 2021 10:14:17 GMT

It's more of a question for rsyslog mailing list, not exactly a splunk one 😉 But we'll see...

One caveat - if you use my supplied config as pasted into an existing config on your system you might get some issues with local logs interfering with received ones. If you use the config as the only configuration directives, you might lose local logging ability. You might simply want to run two different syslog daemons - system-supplied one for local logging and a collecting one for... well, collecting events from remote systems. Your mileage may vary.

The beginning is quite easy to translate. Just do

module(load="imptcp")
input(type="imptcp" port="514")

I'm not sure what the parameters in syslog-ng tcp input definition do, so you might want to tweak some parameters to the input call according to https://www.rsyslog.com/doc/v8-stable/configuration/modules/imptcp.html but for starters this will do.

Then you have the "filer and write" part of your config.

template(name="kernfile" type="string" string="/var/log/syslog-to-splunk/%hostname%/kernel.log")
template(name="mailfile" type="string" string="/var/log/syslog-to-splunk/%hostname%/mail.log")
[...]
template(name="defaultname" type="string" string="/var/log/syslog-to-splunk/%hostname%/program/%programname%.log")

Then you'd only have to do

kern.* action(type="omfile" dynafile="kernfile")
mail.* action(type="omfile" dynafile="mailfile")

And so on. In order to match multiple facilities, you separate them with a comma, so you can do it like that:

auth,authpriv,security.* action(...)

The only other trick we have to pull here is the last - default - rule which should match only those facilities that haven't been matched earlier. So we have to explicitly match only severity "none" in those facilities.

*.*;auth, authpriv, cron, kern, mail, user, security, syslog.none action(type="omfile" dynafile="defaultfile")

Mind you, that's a relatively simple rsyslog config. You can do so much more with rsyslog. Enrich your events, filter them and even send them straight to HEC.

Re: syslog-ng to rsyslog

dmed — Sun, 28 Nov 2021 13:23:38 GMT

Thanks so much for the very informative reply.

I'll try out your suggestions which I'm sure will do the trick 🙂

My setup is such that Logs come in from external clients and all these (including the logs of the local rsyslog server itself) go to a splunkforwarder. Well for now it's just the audit logs that are forwarded, but all logs including those of the log server are collected under directories.

So in effect the local logs per se, are also going to land up under a directory under syslog-to-splunk.

Your config makes good sense to me. Once again, thanks for that. I'll try it out and update here.

Re: syslog-ng to rsyslog

dmed — Fri, 03 Dec 2021 13:51:01 GMT

Hi PickleRick, Sorry for the delay. I just want to confirm your config is working fine for me. I appreciate your help.

I'll start using the rsyslog forums at https://thwack.solarwinds.com/product-forums/loggly/ and / or rsyslog.com forums ( if they even exist 🙂 ) going forward...

Thanks again!

Re: syslog-ng to rsyslog

PickleRick — Fri, 03 Dec 2021 14:33:05 GMT

Solarwinds have nothing to do with rsyslog.

Rsyslog has a good old-fashioned mailing list. https://lists.adiscon.net/mailman/listinfo/rsyslog

Unfortunately, for some reason you have to subscribe to see archives.