Getting Data In

Rsyslog Log Rotation

dillardo_2
Path Finder

We need to rotate syslog files once they reach a certain size.
Our directory structure looks like the following:

/opt/splunk/syslog/datasource1
/opt/splunk/syslog/datasource2
/opt/splunk/syslog/datasource3

etc.

Once files in the source folders reach 1GB for example, we need them to be moved to /opt/splunk/old_files

Has anyone successfully set up a rotation script?

0 Karma

jawahir007
Path Finder

dillardo_2
Path Finder

Thanks, I did see this and have tried using logrotate. When we rotate the logs, they need to be moved to an archive folder outside of the syslog directory. When I add a command to move the files after postrotate, they files are moved successfully, but rsyslog stops working and must be restarted.

For example: 

/opt/splunk/syslog/*/*
 {
    daily
    missingok
    compress    
    notifempty    
    sharedscripts
    postrotate
                mv /opt/splunk/syslog/*/*.gz /opt/splunk/archive/;
    endscript
}
0 Karma

dillardo_2
Path Finder

This works for us, but rsyslog stops processing data and has to be restarted.

0 Karma

mydog8it
Builder

We create a new file per device type every hour and throw away files in 72 hours.

0 Karma

dillardo_2
Path Finder

@mydog8it, are you able to share your process and the script you are using? That would be helpful.

0 Karma

maraman_splunk
Splunk Employee
Splunk Employee

the hourly rotation is by design is you name the log file in rsyslog with mylog-yearmonthday-hour.log
(and that avoid any race condition as it is done directly by rsyslog)
to purge, just add a simple script in /etc/cron.d/purgemylog.cron
with hourly + run as a user who can delete log then run
find /var/log/mylogdir -type f -name \"mylog*.log\"-mtime +2 -delete
if you want to keep them 2 days
(or -mmin +xxx for more granularity)

make sure you specify the directory and a filename form in your find command to avoid any bad surprise...

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...