How to search buckets to find one index of many, and limit how many to unfreeze?

[stevenshea]

After searching the answered questions, I do not see my question addressed.

If I have several indexes that are frozen to buckets, is there a way to search the buckets for an index without unfreezing the bucket?

I'm trying to limit the number of buckets that I need to unfreeze.

Thank you.

[richgalloway · Accepted solution]

I think we have a slight terminology problem. Indexes store data in buckets. Buckets are a subset of an index, therefore, one does not search buckets for an index.

There is no way to search a frozen bucket. You must thaw it first.

While a bucket is part of an index, there is nothing in the bucket itself that identifies the index to which it belongs. That means frozen buckets must be managed in such a way as to know from whence they came. For example, put all of the index=foo buckets in a "foo" directory in your repository. Resist the temptation to dump all frozen Splunk buckets into the same S3 bucket. Otherwise, you'll have to thaw a LOT of buckets to find the one from the desired index.

Once you know the index you need thawed, you need to determine the time frame of the data you want thawed. Buckets are named with Linux epoch timestamps in the form __. Convert the earliest and latest dates of the data you want thawed into epoch form (See https://www.epochconverter.com/) then check the bucket names to find the one(s) that hold data for those dates.

[stevenshea]

Newbie here... thanks for the terminology correction. I did not design the way the frozen buckets were handled but you answered my question. We do not know what buckets are from what index so we have to thaw a lot of buckets.

[richgalloway]

I'm sorry to hear that. Start by narrowing down the time range so you can cut the number of buckets you need to thaw.

If your original question is answered then please accept the answer to help future readers.