Getting Data In

logging all linux logs to syslogs ng for heavy forwarder

ranjitbrhm1
Communicator

Hello All,
I need to send all the LOG files from the client on to the server which is running syslog NG. now whats working for me is all the client logs gets written on the server on to the file that i have mentioned on the syslog conf. But I am wondering now for instance lets say i am not interested in syslog and i am only interested in Apache, messages, dmesg, auth logs etc how do i write the logs different logs on to different files on the syslog machine. So that the final output on Syslog NG server will be /var/log/syslog getting only syslog logs, /var/log/http receiving all the Apache logs etc etc etc and not jumbled up into one single file.
Thanks

0 Karma
1 Solution

p_gurav
Champion
0 Karma

gjanders
SplunkTrust
SplunkTrust

I also have a splunk answers post here on configuring syslogNG settings, this may help you understand how you can change your syslogNG config to send data to separate files.
In my example I accept the data on separate incoming ports to keep the config simple!

0 Karma

ranjitbrhm1
Communicator

Im just quoting from the website here. This is an absolute peach.

 # The default action of syslog-ng is to log a STATS line
 # to the file every 10 minutes. That's pretty ugly after a while.
 # Change it to every 12 hours so you get a nice daily update of
 # how many messages syslog-ng missed (0).
 stats_freq(43200);
 };
source src {
 unix-stream("/dev/log" max-connections(256));
 internal();
 };
source kernsrc { file("/proc/kmsg"); };
# define destinations
 destination authlog { file("/var/log/auth.log"); };
 destination syslog { file("/var/log/syslog"); };
 destination cron { file("/var/log/cron.log"); };
 destination daemon { file("/var/log/daemon.log"); };
 destination kern { file("/var/log/kern.log"); };
 destination lpr { file("/var/log/lpr.log"); };
 destination user { file("/var/log/user.log"); };
 destination mail { file("/var/log/mail.log"); };
 destination mailinfo { file("/var/log/mail.info"); };
 destination mailwarn { file("/var/log/mail.warn"); };
 destination mailerr { file("/var/log/mail.err"); };
 destination newscrit { file("/var/log/news/news.crit"); };
 destination newserr { file("/var/log/news/news.err"); };
 destination newsnotice { file("/var/log/news/news.notice"); };
 destination debug { file("/var/log/debug"); };
 destination messages { file("/var/log/messages"); };
 destination console { usertty("root"); };
# By default messages are logged to tty12...
 destination console_all { file("/dev/tty12"); };
# ...if you intend to use /dev/console for programs like xconsole
 # you can comment out the destination line above that references /dev/tty12
 # and uncomment the line below.
 #destination console_all { file("/dev/console"); };
# create filters
 filter f_authpriv { facility(auth, authpriv); };
 filter f_syslog { not facility(authpriv, mail); };
 filter f_cron { facility(cron); };
 filter f_daemon { facility(daemon); };
 filter f_kern { facility(kern); };
 filter f_lpr { facility(lpr); };
 filter f_mail { facility(mail); };
 filter f_user { facility(user); };
 filter f_debug { not facility(auth, authpriv, news, mail); };
 filter f_messages { level(info..warn)
 and not facility(auth, authpriv, mail, news); };
 filter f_emergency { level(emerg); };
 filter f_info { level(info); };
 filter f_notice { level(notice); };
 filter f_warn { level(warn); };
 filter f_crit { level(crit); };
 filter f_err { level(err); };
 filter f_failed { message("failed"); };
 filter f_denied { message("denied"); };

# connect filter and destination
log { source(src); filter(f_authpriv); destination(authlog); };
log { source(src); filter(f_syslog); destination(syslog); };
log { source(src); filter(f_cron); destination(cron); };
log { source(src); filter(f_daemon); destination(daemon); };
log { source(kernsrc); filter(f_kern); destination(kern); };
log { source(src); filter(f_lpr); destination(lpr); };
log { source(src); filter(f_mail); destination(mail); };
log { source(src); filter(f_user); destination(user); };
log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); };
log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn); };
log { source(src); filter(f_mail); filter(f_err); destination(mailerr); };
log { source(src); filter(f_debug); destination(debug); };
log { source(src); filter(f_messages); destination(messages); };
log { source(src); filter(f_emergency); destination(console); };
# default log
log { source(src); destination(console_all); };
0 Karma

p_gurav
Champion

Hi,

For syslogng this post will be helpful:
http://www.monitis.com/blog/how-to-filter-logs-with-syslog-ng/

And if you want to do filtering at Splunk side, refer this doc:
http://docs.splunk.com/Documentation/Splunk/7.0.2/Forwarding/Routeandfilterdatad

0 Karma

p_gurav
Champion

do you need syslog file configuration or how you can separate it in Splunk?

0 Karma

ranjitbrhm1
Communicator

Syslog configuration would be ideal. but if there is no other option but there is an option in splunk, then that can also be explored as a last resort. My idea is also not to shoot my license. so a syslog option if there are any would be the best.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...