Deployment Architecture

How to delete/overwrite data older than X number of days

ericmoss
Explorer

I'm interested in only keeping data for X number of days. After X days, I want the data to be deleted from Splunk. Is this possible?

Can it be done through the GUI or config files?

Is X equal to number of days? hours? minutes? seconds?

Is it specific to only to one index or globally across all data that Splunk indexes?

ziegfried
Influencer

You can configure a retention period per index and you have to do this in the configuration file (indexes.conf).

The configuration option is called frozenTimePeriodInSecs and has to be configured in seconds. So for example if you want to keep your events for 30 days, you would configure it this way:

[test_index]
homePath = $SPLUNK_DB/test_index/db
coldPath = $SPLUNK_DB/test_index/colddb
thawedPath = $SPLUNK_DB/test_index/thaweddb
frozenTimePeriodInSecs = 2592000

Note that splunk will not remove every event once it's older than defined in frozenTimePeriodInSecs, but when the bucket it's contained in is past this period. See http://www.splunk.com/base/Documentation/latest/Admin/Setaretirementandarchivingpolicy

jaredlaney
Contributor

@ziegfried - If I push a file to splunk once a day with the above configuration with _time events up to 60 days old, when will they be removed? How do these events get organized by bucket?

example:
test_index_10_27_2015.csv (events over the past 60 days)
test_index_10_26_2015.csv (events over the past 60 days)
test_index_10_25_2015.csv (events over the past 60 days)
test_index_10_24_2015.csv (events over the past 60 days)

0 Karma

jaredlaney
Contributor

Also, imagine each file is pushed once per day and ingested by the forwarder.

0 Karma
Get Updates on the Splunk Community!

Introducing Ingest Actions: Filter, Mask, Route, Repeat

WATCH NOW Ingest Actions (IA) is the best new way to easily filter, mask and route your data in Splunk® ...

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...