Deployment Architecture

Indexes having more data than retention period defined

ramprakash
Explorer

Hi Everyone,

We have one index defined in indexes.conf with frozenTimePeriodInSecs as 365 days (31536000 seconds), but there are 3 years of data stored in index.It seems not working if we just define retention time period in frozenTimePeriodInSecs. Can someone help ?

$ view ./apps/launcher/local/indexes.conf
[Indexname]
coldPath = $SPLUNK_DB/Indexname/colddb
homePath = $SPLUNK_DB/Indexname/db
thawedPath = $SPLUNK_DB/Indexname/thaweddb

Maximum index total size in MB

maxTotalDataSizeMB = 35000

frozenTimePeriodInSecs = 31536000

0 Karma

somesoni2
Revered Legend

Could you run this search and see what values of startEpoch and endEpoch you get for your index.

| dbinspect index=YourIndex earliest=0 | table index *Epoch splunk_server path  | convert ctime(*Epoch) | rename splunk_server as Indexer

This command will list all the data buckets you have for your index. If endEpoch values are newer than your data retention period, then those buckets will not be frozen.

0 Karma

ramprakash
Explorer

Hi@somesoni2..I think the issue is with hot buckets only. All the data is in Hot and Warm buckets and nothing has been moved to Cold buckets. Can you guide me what best config i choose so that 365 days data is always searchable and old data should be delete.

0 Karma

ramprakash
Explorer

I pulled data from Search head and found that it has 3 years of data but when i logged in to check configuration files it shows me 1 year retention settings

0 Karma

skoelpin
SplunkTrust
SplunkTrust

To clarify, you pulled this from your indexer and not the search head right?

0 Karma

ramprakash
Explorer

I pulled data from Search head and found that it has 3 years of data but when i logged in to check configuration files it shows me 1 year retention settings

0 Karma

skoelpin
SplunkTrust
SplunkTrust

The stanza you posted in your original question, did that stanza come from the search head or indexer?

0 Karma

ramprakash
Explorer

I pulled data from Search head and found that it has 3 years of data but when i logged in to check configuration files it shows me 1 year retention settings

0 Karma

skoelpin
SplunkTrust
SplunkTrust

What server did you login to check the configuration files? If you logged into the search head, then this will have no impact on retention as data lives on the indexers.

Is this a standalone setup?

0 Karma

ramprakash
Explorer

No I checked the Indexer configuration only. Yes it is standalone setup

0 Karma

skoelpin
SplunkTrust
SplunkTrust

I don't understand... How could you check the indexer config only if this is a standalone setup?

0 Karma

ramprakash
Explorer

Sorry i meant it is not clustered environment

0 Karma

lakshman239
SplunkTrust
SplunkTrust

If you define the changes via the config, you will need to restart the splunk, to make effect.

0 Karma

ramprakash
Explorer

Yes Splunk has been restarted and it has most recent data in Indexes.

0 Karma

lakshman239
SplunkTrust
SplunkTrust

ok. As a good practice, pls put all your config in a custom 'app' or under 'search' app if its temporary. If the issue is resolved, pls accept the answer to close the thread.

0 Karma

somesoni2
Revered Legend

Also, the data (bucket) is frozen only when the most recent event in the bucket is older then the retention period. Sometimes, a bucket can have data for varying/larger dates-range (e.g. a bucket has data for whole 1 year) and doesn't roll until the event with newest time is older than retention period. See this for more information on the same.

https://docs.splunk.com/Documentation/Splunk/7.2.3/Indexer/Setaretirementandarchivingpolicy#Freeze_d...

0 Karma

ramprakash
Explorer

Do i need to check configuration file precedence if the settings mentioned by me are fine.

0 Karma

somesoni2
Revered Legend

Sure. Run btool command on your indexers to see what frozenTimePeriodInSecs is effective.

Go to $SPLUNK_HOME/bin and then run this:

./splunk btool indexes list IndexName --debug

This should show you what the effective configuration is and from what location.

0 Karma

ramprakash
Explorer

Yes it is showing 365 days only

0 Karma

Tune In & Win!

Don't miss out on your
chance to take home free
prizes by helping our players
save the Splunk Cloudom!

Dungeons & Data
Monsters: Splunk O11y
Day Editions Games
stream live:
5/4 at 6:30pm PST
5/5 at 7:00pm PST
on