Getting Data In

Is there a way to delete data on an indexer from Splunk Web to free up disk space?

Abilan1
Path Finder

Hi ,

We are about to reach the maximum size of the disk on our Indexer server. Please suggest if there is any way to purge data that is 90 days old from Splunk Web, or is there any setting to overwrite the old files from Splunk Web?

0 Karma
1 Solution

lguinn2
Legend

Here are the ways that you can delete data from your index: Remove indexes and indexed data

The delete command will remove data from an index, but does not recover the disk space.
Using splunk clean to clean an index will remove all data and recover the disk space; there is no way to do this by date.

You can set retention time for an index in indexes.conf by using the frozenTimePeriodInSecs setting

[yourindex]
#other index settings
frozenTimePeriodInSecs = 31556926

31556926 is the number of seconds in a year. You will need to restart Splunk for this setting to take effect. Splunk will immediately began to remove data older than one year from the index.

View solution in original post

lguinn2
Legend

Here are the ways that you can delete data from your index: Remove indexes and indexed data

The delete command will remove data from an index, but does not recover the disk space.
Using splunk clean to clean an index will remove all data and recover the disk space; there is no way to do this by date.

You can set retention time for an index in indexes.conf by using the frozenTimePeriodInSecs setting

[yourindex]
#other index settings
frozenTimePeriodInSecs = 31556926

31556926 is the number of seconds in a year. You will need to restart Splunk for this setting to take effect. Splunk will immediately began to remove data older than one year from the index.

lguinn2
Legend

Go into the settings menu on the indexer, and look for indexes. On the indexes page, you can see the size of the various indexes. Reduce the size of one or more indexes. Splunk will immediately begin to remove/freeze the oldest data until all indexes are under the maximum size.

You can also set a time limit for the data as well, but that setting is not available from the GUI.

Abilan1
Path Finder

Thank You so much!
1. In case If I want to clean up all the 1 year old Data from my "TEST" index, how can I do that?
2. Also please confirm me we need to modify which size (Max size (MB) of entire index or Max size (MB) of hot/warm/cold bucket)? Also If we modified this size, will it automatically purge the old data in future whenever this index touches it's maximum size. do we need to restart after this modification?

0 Karma

Abilan1
Path Finder

Hi ,

Can someone please help me here?

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Have a look at this https://wiki.splunk.com/Deploy:BucketRotationAndRetention

The data retention is done either based on total max size of index (hot + warm + col) using maxTotalDataSizeMB OR based on bucket age (buckets older than specified period will be frozen(deleted by default)) using frozenTimePeriodInSecs attribute, for the index in indexes.conf.

The purging will done automatically by Splunk at regular interval and a restart will be required for this change to take place (if done via conf files directly).

0 Karma

Abilan1
Path Finder

Thank you!

0 Karma

Abilan1
Path Finder

Hi ,

I have updated the settings as like below and restarted splunk, but it didn't clean up my old data from the indexer. Please find my indexes.conf below

[test]
coldpath = $SPLUNKDB/test/colddb
homepath = $SPLUNKDB/test/db
thawedpath = $SPLUNKDB/test/thaweddb
maxTotalDataSizeMB = 500000
frozenTimePeriodInSecs = 31556926

Please let me know if you still want to make any changes in the index details.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...