Getting Data In

Question About Retetion Policy No Effect

aojie654
Path Finder

Hi, Splunkers:

I have a question about retention policy that I had configured my index linux_log of frozenTimePeriodInSecs in stanza default in indexes.conf, then the result after splunk btool as following:

# splunk btool --debug indexes list linux_log | grep frozenTimePeriodInSecs
/opt/splunk/etc/system/local/indexes.conf      frozenTimePeriodInSecs = 31624400

but I saw the index info of linux_log in monitoring console like this:
alt text

Thats obvious that there are some data is older than retention settings! And I can searching the retention data also:
alt text

The timestamp of first event is the time when I started this server... That is, no data was deleted from splunk even they were older than retention limit.

In actually, we had configured the retention limit to 1 year in my customer's splunk but there are 6 years' data, and their disk will be fill full in next 3 days.

There are 3 questions:
1. Why is this?
2. How can I delete the old data? I know that I can identify the timestamp then delete the bucket folder in db path, but the timestamp in bucket name is index_time but not event_timestamp, isn't it?
3. I can't using delete in a search for the command is only remove the event from search result and will not clean up the disk, right?

0 Karma
1 Solution

woodcock
Esteemed Legend

Buckets contain a span of data. In order for a bucket to be frozen (deleted), the newest event in the bucket must be older than frozenTimePeriodInSecs. For this reason, people often constrain buckets to be at most 1-day wide but this can cause very serious side-effects, including scalability problems that are unfixable.

View solution in original post

woodcock
Esteemed Legend

Buckets contain a span of data. In order for a bucket to be frozen (deleted), the newest event in the bucket must be older than frozenTimePeriodInSecs. For this reason, people often constrain buckets to be at most 1-day wide but this can cause very serious side-effects, including scalability problems that are unfixable.

aojie654
Path Finder

Ok, thanks for you reply, I'll delete some bucket event we will lost some data.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...