Getting Data In

How do I delete old log data past a certain time on an index?

bayman
Path Finder

We're running out of disk space.

How do I delete old log data past a certain time on an index?

If I set a max index size, what happens when that limit is reached for an index?

How should I rotate logs so old logs are automatically deleted?

0 Karma
1 Solution

skoelpin
SplunkTrust
SplunkTrust

You can delete data directly from the file system or use the clean command

$SPLUNK_HOME/bin/splunk clean eventdata -index <index_name>

Note that the delete command will not delete data from the file system, it will only hide it in Splunk web

http://docs.splunk.com/Documentation/Splunk/6.1.4/Indexer/RemovedatafromSplunk

If you set max index size, then the oldest data that is past that max size will either be deleted or archived if you specified a frozen path when creating your index.

Splunk buckets will roll from hot --> warm --> cold --> frozen.. I believe by default they will roll to frozen every 6 years OR until they reach the max index size

http://docs.splunk.com/Documentation/Splunk/6.5.2/Indexer/HowSplunkstoresindexes

View solution in original post

skoelpin
SplunkTrust
SplunkTrust

You can delete data directly from the file system or use the clean command

$SPLUNK_HOME/bin/splunk clean eventdata -index <index_name>

Note that the delete command will not delete data from the file system, it will only hide it in Splunk web

http://docs.splunk.com/Documentation/Splunk/6.1.4/Indexer/RemovedatafromSplunk

If you set max index size, then the oldest data that is past that max size will either be deleted or archived if you specified a frozen path when creating your index.

Splunk buckets will roll from hot --> warm --> cold --> frozen.. I believe by default they will roll to frozen every 6 years OR until they reach the max index size

http://docs.splunk.com/Documentation/Splunk/6.5.2/Indexer/HowSplunkstoresindexes

bayman
Path Finder

/opt/splunk/var/lib/splunk folder size is 200G of data.
I'm assuming I manage this folder size via the Index size limit?

/var/log/splunk folder size is 90G of data
How should I manage this folder size? Is it safe to delete these *.log files in this folder?

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Yes correct, by default the max index size will be 500GB. Go to Settings>Index and find your index and modify the size limits

I'm assuming var/log/splunk is on a separate server which is being forwarded to Splunk? If so then yeah you can delete those log files as it's already been ingested by Splunk (Check before removing! A better strategy would be to zip them or move to another drive if they are important). As for log rotation, that's more of a sys-admin task rather than a Splunk task. You will either need to grow the drive or roll your logs on a regular basis

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...