topic Re: Rsyslog Log Rotation in Getting Data In

Rsyslog Log Rotation

dillardo_2 — Fri, 24 Jan 2020 16:37:45 GMT

We need to rotate syslog files once they reach a certain size.
Our directory structure looks like the following:

/opt/splunk/syslog/datasource1
/opt/splunk/syslog/datasource2
/opt/splunk/syslog/datasource3

etc.

Once files in the source folders reach 1GB for example, we need them to be moved to /opt/splunk/old_files

Has anyone successfully set up a rotation script?

Re: Rsyslog Log Rotation

mydog8it — Fri, 24 Jan 2020 19:32:47 GMT

We create a new file per device type every hour and throw away files in 72 hours.

Re: Rsyslog Log Rotation

dillardo_2 — Sun, 26 Jan 2020 20:35:48 GMT

@mydog8it, are you able to share your process and the script you are using? That would be helpful.

Re: Rsyslog Log Rotation

maraman_splunk — Sun, 26 Jan 2020 21:20:11 GMT

the hourly rotation is by design is you name the log file in rsyslog with mylog-yearmonthday-hour.log
(and that avoid any race condition as it is done directly by rsyslog)
to purge, just add a simple script in /etc/cron.d/purgemylog.cron
with hourly + run as a user who can delete log then run
find /var/log/mylogdir -type f -name \"mylog*.log\"-mtime +2 -delete
if you want to keep them 2 days
(or -mmin +xxx for more granularity)

make sure you specify the directory and a filename form in your find command to avoid any bad surprise...

Re: Rsyslog Log Rotation

Jawahir — Mon, 27 Jan 2020 12:27:04 GMT

Refer this link:
https://www.tecmint.com/manage-linux-system-logs-using-rsyslogd-and-logrotate/

Re: Rsyslog Log Rotation

dillardo_2 — Mon, 27 Jan 2020 23:40:19 GMT

Thanks, I did see this and have tried using logrotate. When we rotate the logs, they need to be moved to an archive folder outside of the syslog directory. When I add a command to move the files after postrotate, they files are moved successfully, but rsyslog stops working and must be restarted.

For example: 

/opt/splunk/syslog/*/*
 {
    daily
    missingok
    compress    
    notifempty    
    sharedscripts
    postrotate
                mv /opt/splunk/syslog/*/*.gz /opt/splunk/archive/;
    endscript
}

Re: Rsyslog Log Rotation

dillardo_2 — Mon, 27 Jan 2020 23:42:59 GMT

This works for us, but rsyslog stops processing data and has to be restarted.