Why is thawed data not working correctly or missing after restoring frozen data?

[hagjos43]

In a test environment (two indexers, one SH, one cluster master/deployment server) I froze any data that was older than half a day.
Indexes.conf:

[endor]
repFactor = auto
homePath   = $SPLUNK_DB\endor\db
coldPath   = $SPLUNK_DB\endor\colddb
thawedPath = $SPLUNK_DB\endor\thaweddb
maxWarmDBCount = 2
maxDataSize = auto_high_volume
frozenTimePeriodInSecs = 43200
coldToFrozenDir = $SPLUNK_HOME\Archive\endor

Then I used the rebuild method on both indexers following directions here: docs.splunk.com/Documentation/Splunk/6.3.1/Indexer/Restorearchiveddata

The rebuild did not restore all of the data and only allows me to search back to December 17th, prior to the frozen bucket configuration the data went back to April 2014ish.

[jkat54]

You have to drop into thawed, then rebuild the buckets, then restart splunk on each indexer.

It can often be done with a simple for loop in shell.

I don’t understand why that process didn’t work for the operator, but I don’t accept the answer I gave in 2015 and believe that was wrong.

[jkat54 · Accepted solution]

If you're "thawing" data older than frozenTimePeriodInSecs, it will probably go right back into the frozen directory.

Try creating a new index with defaults and only thaw your previous index into it's thawed directory.

With your original setup, you might find more frozen buckets in $SPLUNK_HOME\Archive\endor (your coldToFrozenDir) as soon as you drop them in thawed and run the rebuild command.

Added after acceptance:
Just to make it clear... hagjos was thawing data back into the same index it was frozen out of. So when he did this, the new thawed data was being detected as older than frozenTimePeriodInSecs and dropping right out of splunk into the frozen directory again.

By creating a new index and thawing into it, he circumvented the immediate freezing of his old data because his new index has a default frozenTimePeriodInSecs of 6 years.

[jkat54]

Honestly after rereading and thought I'm thinking if you drop into thawed, it doesn't get moved to frozen, it stays in thawed.  Not sure how this answer worked.

[splunkreal]

Hello, you are right, if you put files in thawed it does return to frozen, you have to delete manually.

[hagjos43]

I placed the data(buckets) from the frozen directory into thawedPath = $SPLUNK_DB\endor\thaweddb then ran the rebuild, shouldn't that have worked? or do I need to still create a new index?

[hagjos43]

I followed your instructions and created a new index and it worked perfectly.

Thank you!

[jkat54]

Thanks for accepting the answer.