Does anybody know how to set log retention in syslog-ng for 2 weeks

[rjrijo]

Hi All

i have configured syslog-ng in our company to collect the logs form network devices,I am little bit confuse for setting the syslog server retention 

options {
chain_hostnames(no);
create_dirs (yes);
dir_perm(0755);
dns_cache(yes);
keep_hostname(yes);
log_fifo_size(2048);
log_msg_size(8192);
perm(0644);
time_reopen (10);
use_dns(yes);
use_fqdn(yes);
};

source s_network {
tcp(ip(0.0.0.0) port(514));
udp(ip(0.0.0.0) port(514));
};

#Destinations
destination d_all { file("/var/syslog/logs/catch_all/$HOST/$YEAR-$MONTH-$DAY-$HOUR-catch_all.log" perm(0666) dir_perm(0777) create_dirs(yes)); };
log { source(s_network); destination(d_all); };

# Source additional configuration files (.conf extension only)
#@include "/etc/syslog-ng/conf.d/*.conf"

# vim:ft=syslog-ng:ai:si:ts=4:sw=4:et:
# command line audit logging
local1.* -/var/log/cmdline

 Please see the above configuration and advise?

Spoiler

• splunk-enterprise
• syslog
• syslog-ng
• syslogs

[zacharychristen]

Syslog-ng does not handle log rotation. This can be handled through cron or the logrotate command.

To use logrotate you will need to add a configuration file to /etc/logrotate.d. For example you could create a file called "splunk_logs" in /etc/logrotate.d. For simplicity, you can copy an existing file in /etc/logrotate.d and use it as a template. You will then want to modify to fit your use case (see example below).

/var/log/syslog/logs/catch_all/*.log   # Replace with the correct Path
{  
    rotate 2  # Keep 2 Files. This combined with the following will keep logs for 2 weeks.
    weekly    # Rotate Weekly
    missingok
    notifempty
    compress
    sharedscripts
    postrotate
        [invoke-rc.d syslog-ng reload > /dev/null]
    endscript
}

See the man pages for logrotate to learn more.

To use cron you could do something like this to keep the logs for 2 weeks:

run crontab -e as root then add the following cronjob.

0 3 * * * /usr/bin/find /var/log/syslog/logs/catch_all -type f -daystart -mtime +14 -exec rm {} \;

This will run at 3 am everyday and remove the logs after 14 days. Be sure to update the path to the log files.

Hope this helps! Happy Splunking!

[gcusello]

Hi @rjrijo,

retention is defined in the index definition, so you have to add (in indexes.conf) to the index where you're storing these logs the option (for 15 days):

frozenTimePeriodInSecs = 1296000

as described at https://docs.splunk.com/Documentation/Splunk/8.0.6/Indexer/Setaretirementandarchivingpolicy

Ciao.

Giuseppe