Getting Data In

Does anybody know how to set log retention in syslog-ng for 2 weeks

rjrijo
Engager

 

Hi All

i have configured syslog-ng in our company to collect the logs form network devices,I am little bit confuse for setting the syslog server retention 

 
 

options {
chain_hostnames(no);
create_dirs (yes);
dir_perm(0755);
dns_cache(yes);
keep_hostname(yes);
log_fifo_size(2048);
log_msg_size(8192);
perm(0644);
time_reopen (10);
use_dns(yes);
use_fqdn(yes);
};


source s_network {
tcp(ip(0.0.0.0) port(514));
udp(ip(0.0.0.0) port(514));
};


#Destinations
destination d_all { file("/var/syslog/logs/catch_all/$HOST/$YEAR-$MONTH-$DAY-$HOUR-catch_all.log" perm(0666) dir_perm(0777) create_dirs(yes)); };
log { source(s_network); destination(d_all); };


# Source additional configuration files (.conf extension only)
#@include "/etc/syslog-ng/conf.d/*.conf"


# vim:ft=syslog-ng:ai:si:ts=4:sw=4:et:
# command line audit logging
local1.* -/var/log/cmdline

 

 

 Please see the above configuration and advise?

Spoiler

 

Labels (1)

zacharychristen
Path Finder

Syslog-ng does not handle log rotation. This can be handled through cron or the logrotate command.

To use logrotate you will need to add a configuration file to /etc/logrotate.d. For example you could create a file called "splunk_logs" in /etc/logrotate.d. For simplicity, you can copy an existing file in /etc/logrotate.d and use it as a template. You will then want to modify to fit your use case (see example below).

/var/log/syslog/logs/catch_all/*.log   # Replace with the correct Path
{
rotate 2 # Keep 2 Files. This combined with the following will keep logs for 2 weeks. weekly # Rotate Weekly missingok notifempty compress sharedscripts postrotate [invoke-rc.d syslog-ng reload > /dev/null] endscript
}

See the man pages for logrotate to learn more.

To use cron you could do something like this to keep the logs for 2 weeks:

run crontab -e as root then add the following cronjob.

0 3 * * * /usr/bin/find /var/log/syslog/logs/catch_all -type f -daystart -mtime +14 -exec rm {} \;

This will run at 3 am everyday and remove the logs after 14 days. Be sure to update the path to the log files.

Hope this helps! Happy Splunking!

0 Karma

gcusello
Esteemed Legend

Hi @rjrijo,

retention is defined in the index definition, so you have to add (in indexes.conf) to the index where you're storing these logs the option (for 15 days):

 

frozenTimePeriodInSecs = 1296000

 

as described at https://docs.splunk.com/Documentation/Splunk/8.0.6/Indexer/Setaretirementandarchivingpolicy

Ciao.

Giuseppe 

0 Karma
Get Updates on the Splunk Community!

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...

Security Highlights | January 2023 Newsletter

January 2023 Splunk Security Essentials (SSE) 3.7.0 ReleaseThe free Splunk Security Essentials (SSE) 3.7.0 app ...

Platform Highlights | January 2023 Newsletter

 January 2023Peace on Earth and Peace of Mind With Business ResilienceAll organizations can start the new year ...