How to delete/overwrite data older than X number of days

[ericmoss]

I'm interested in only keeping data for X number of days. After X days, I want the data to be deleted from Splunk. Is this possible?

Can it be done through the GUI or config files?

Is X equal to number of days? hours? minutes? seconds?

Is it specific to only to one index or globally across all data that Splunk indexes?

[ziegfried]

You can configure a retention period per index and you have to do this in the configuration file (indexes.conf).

The configuration option is called frozenTimePeriodInSecs and has to be configured in seconds. So for example if you want to keep your events for 30 days, you would configure it this way:

[test_index]
homePath = $SPLUNK_DB/test_index/db
coldPath = $SPLUNK_DB/test_index/colddb
thawedPath = $SPLUNK_DB/test_index/thaweddb
frozenTimePeriodInSecs = 2592000

Note that splunk will not remove every event once it's older than defined in frozenTimePeriodInSecs, but when the bucket it's contained in is past this period. See http://www.splunk.com/base/Documentation/latest/Admin/Setaretirementandarchivingpolicy

[jaredlaney]

@ziegfried - If I push a file to splunk once a day with the above configuration with _time events up to 60 days old, when will they be removed? How do these events get organized by bucket?

example:
test_index_10_27_2015.csv (events over the past 60 days)
test_index_10_26_2015.csv (events over the past 60 days)
test_index_10_25_2015.csv (events over the past 60 days)
test_index_10_24_2015.csv (events over the past 60 days)

[jaredlaney]

Also, imagine each file is pushed once per day and ingested by the forwarder.