Indexes having more data than retention period defined

[ramprakash]

Hi Everyone,

We have one index defined in indexes.conf with frozenTimePeriodInSecs as 365 days (31536000 seconds), but there are 3 years of data stored in index.It seems not working if we just define retention time period in frozenTimePeriodInSecs. Can someone help ?

$ view ./apps/launcher/local/indexes.conf
[Indexname]
coldPath = $SPLUNK_DB/Indexname/colddb
homePath = $SPLUNK_DB/Indexname/db
thawedPath = $SPLUNK_DB/Indexname/thaweddb

Maximum index total size in MB

maxTotalDataSizeMB = 35000

frozenTimePeriodInSecs = 31536000

[somesoni2]

Could you run this search and see what values of startEpoch and endEpoch you get for your index.

| dbinspect index=YourIndex earliest=0 | table index *Epoch splunk_server path  | convert ctime(*Epoch) | rename splunk_server as Indexer

This command will list all the data buckets you have for your index. If endEpoch values are newer than your data retention period, then those buckets will not be frozen.

[ramprakash]

Hi@somesoni2..I think the issue is with hot buckets only. All the data is in Hot and Warm buckets and nothing has been moved to Cold buckets. Can you guide me what best config i choose so that 365 days data is always searchable and old data should be delete.

[ramprakash]

I pulled data from Search head and found that it has 3 years of data but when i logged in to check configuration files it shows me 1 year retention settings

[skoelpin]

To clarify, you pulled this from your indexer and not the search head right?

[ramprakash]

I pulled data from Search head and found that it has 3 years of data but when i logged in to check configuration files it shows me 1 year retention settings

[skoelpin]

The stanza you posted in your original question, did that stanza come from the search head or indexer?

[ramprakash]

I pulled data from Search head and found that it has 3 years of data but when i logged in to check configuration files it shows me 1 year retention settings

[skoelpin]

What server did you login to check the configuration files? If you logged into the search head, then this will have no impact on retention as data lives on the indexers.

Is this a standalone setup?

[ramprakash]

No I checked the Indexer configuration only. Yes it is standalone setup

[skoelpin]

I don't understand... How could you check the indexer config only if this is a standalone setup?

[ramprakash]

Sorry i meant it is not clustered environment

[lakshman239]

If you define the changes via the config, you will need to restart the splunk, to make effect.

[ramprakash]

Yes Splunk has been restarted and it has most recent data in Indexes.

[lakshman239]

ok. As a good practice, pls put all your config in a custom 'app' or under 'search' app if its temporary. If the issue is resolved, pls accept the answer to close the thread.

[somesoni2]

Also, the data (bucket) is frozen only when the most recent event in the bucket is older then the retention period. Sometimes, a bucket can have data for varying/larger dates-range (e.g. a bucket has data for whole 1 year) and doesn't roll until the event with newest time is older than retention period. See this for more information on the same.

https://docs.splunk.com/Documentation/Splunk/7.2.3/Indexer/Setaretirementandarchivingpolicy#Freeze_d...

[ramprakash]

Do i need to check configuration file precedence if the settings mentioned by me are fine.

[somesoni2]

Sure. Run btool command on your indexers to see what frozenTimePeriodInSecs is effective.

Go to $SPLUNK_HOME/bin and then run this:

./splunk btool indexes list IndexName --debug

This should show you what the effective configuration is and from what location.

[ramprakash]

Yes it is showing 365 days only