Best practices for hot/warm bucket retention?

[splunker969]

The primary indexers data (Hot+ Warm) data is being full .Please help us in solving this issues . .We are trying to shrink the hot and warm are our primary indexers .
The retention period for hot +warm is 30 days.

What are best practice process? To come out of this issue .

[koshyk]

I'm not sure what you meant by Shrink? That's genuine data
Some of the options I can think of are:
1. Reduce 30days hot+warm and move to cold
2. increase disk space for hot storage

[splunker969]

Thanks! koshyk

[somesoni2]

Data in hot/warm buckets are not managed by Splunk's data retention policy. Data is only frozen (archived or deleted based on what you've configured) from cold bucket directory. If you want to reduce the amount of data you want to retain for your index, ensure that you've configuration to have warm buckets rolled to cold buckets. See this for list of attributes that affect the data buckets:
http://docs.splunk.com/Documentation/Splunk/6.6.3/Indexer/Configureindexstorage#Attributes_that_affe...

Consider (appropriately) updating attributes maxHotSpanSecs, maxDataSize and maxWarmDBCount so that buckets will roll (eventually) to cold bucket.

[Sukisen1981]

Correctly explained in detail by @somesoni2 . The data retention policy of 30 days is not a guarantee that ALL data in a 30 day period will be stored in the hot / wam buckets. What is important is to know how much data you are indexing. Things can get a bit complex here. Your organization will pay Splunk for the maximum amount of data indexed in a 24 hour period, but since you are not receiving any license violation warnings, I assume that is not a concern at this point of time.
@somesoni2 says - Consider (appropriately) updating attributes maxHotSpanSecs, maxDataSize and maxWarmDBCount so that buckets will roll (eventually) to cold bucket.
You need to make a decision on what size (how much GB) of your data is needed in hot and warm buckets.

[splunker969]

Thanks @sukisen1981

[splunker969]

Thanks @somesoni2