Halp! My data is being rolled to frozen and I don't know why!

[bosburn_splunk]

I need to know why my data is being rolled to frozen - is it because of time or disk space?

[bosburn_splunk]

As an update to this (since I've recently had to look into this) - for our Splunk Cloud customers, you can use the following search:

index=_internal source=*splunkd.log host=c0m1* component=CMMasterRemoteStorageThread freezing reason=* 
| convert ctime(bucket_earliest) AS Earliest_Event 
| convert ctime(bucket_latest) AS Latest_Event 
| convert ctime(_time) AS Event_Time 
| table Event_Time,idx,Earliest_Event,Latest_Event,reason

Hope this helps! 

[gjanders]

I also have a few searches for this in Alerts for Splunk Admins or github

For example:
IndexerLevel - Buckets are been frozen due to index sizing

index=_internal sourcetype=splunkd (source=*splunkd.log) "BucketMover - will attempt to freeze" NOT "because frozenTimePeriodInSecs=" 
| rex field=bkt "(rb_|db_)(?P<newestDataInBucket>\d+)_(?P<oldestDataInBucket>\d+)"
| eval newestDataInBucket=strftime(newestDataInBucket, "%+"), oldestDataInBucket = strftime(oldestDataInBucket, "%+") 
| table message, oldestDataInBucket, newestDataInBucket

Note the above alert excludes time based rolling, it detects the issue where you hit the size limits and data rolled to frozen...

[DEAD_BEEF]

I came across this post in my own searches, I used the search from @bosburn_splunk but updated it for my instance (v7.0) and thought I'd share what I used incase the syntax has changed from what was originally posted in 2014.

What you'll need to edit in the SPL is the path within the logs. You'll see the patterns below. I am matching on both patterns, that's why in my query you'll see it starts with '((pattern1)|(pattern2))...

bkt='<path>...'
OR
candidate='<path>...'

My working query

index=_internal sourcetype=splunkd component=BucketMover "Will attempt to freeze" 
| rex field=_raw "'((\/mnt\/local\/(?<bucket>[^\/]+))|(\/opt\/splunk\/var\/lib\/splunk))\/(?<index_name>[^\/]+)\/(db|colddb)\/(db|rb)_(?<latest_epoch>\d+)_(?<earliest_epoch>\d+)_(?<bucket_number>\d+)_(?<guid>[A-Z0-9-]+)' because (?<Reason>.*)"
| convert ctime(earliest_epoch) as Earliest_Data 
| convert ctime(latest_epoch) as Latest_Data 
| convert ctime(_time) as Log_TimeStamp 
| table Log_TimeStamp,index_name,bucket,bucket_number,Earliest_Data,Latest_Data,Reason 
| sort - Log_TimeStamp

I referenced this table for the bucket naming conventions.

[bosburn_splunk · Accepted solution]

With this handy dandy query, you can find out why!

index=_internal sourcetype=splunkd component=BucketMover "Will attempt to freeze" | rex field=_raw "/logs/splunk_logs/(?P<index_name>[^\/]+)/(db|colddb)/db_(?P<earliest_epoch>[\d]+)_(?P<latest_epoch>[\d]+)_(?P<bucket_number>[\d]+)\' (?P<reason>.*)"  | convert ctime(earliest_epoch) as Earliest_Data  | convert ctime(latest_epoch) as Latest_Data  | convert ctime(_time) as Log_TimeStamp | table Log_TimeStamp,index_name,bucket_number,Earliest_Data,Latest_Data,reason | sort - Log_TimeStamp

Breakdown of the fields:

Log_TimeStamp = _time of the log entry
index_name = index that's being frozen
bucket_number = bucket that's being frozen
Earliest_Data = Earliest date of events that was in that bucket
Latest_Data = Latest date of events that was in that bucket
Reason = the reason it was rolled to frozen.

Edit: Addition query for windows!
Just an updated version for windows (since there's wackiness around escaping \'s):

index=_internal sourcetype=splunkd component=BucketMover "Will attempt to freeze" | rex field=_raw "(?P<index_name>[^\\\\]+)\\\\(db|colddb)\\\\db_(?P<earliest_epoch>[\d]+)_(?P<latest_epoch>[\d]+)_(?P<bucket_number>[\d]+)' (?P<reason>.*)"  | convert ctime(earliest_epoch) as Earliest_Data  | convert ctime(latest_epoch) as Latest_Data  | convert ctime(_time) as Log_TimeStamp | table Log_TimeStamp,index_name,bucket_number,Earliest_Data,Latest_Data,reason | sort - Log_TimeStamp

[Hdholariya]

Common search for the linux and windows.

index=_internal sourcetype=splunkd component=BucketMover "Will attempt to freeze" 
| rex field=_raw "(/|\\\)splunk(/|\\\)(?P<index_name>[^\/]+)(/|\\\)(db|colddb)(/|\\\)db_(?P<latest_event>[\d]+)_(?P<earliest_event>[\d]+)_(?P<bucket_number>[^\']+)\' (?P<reason>.*)" 
| convert ctime(earliest_event) as earliest_event 
| convert ctime(latest_event) as latest_event 
| convert ctime(_time) as Log_TimeStamp 
| table Log_TimeStamp,index_name,bucket_number,earliest_event,latest_event,reason 
| sort - Log_TimeStamp

---

[bosburn_splunk]

Just an updated version for windows (since there's wackiness around escaping \'s):

index=internal sourcetype=splunkd component=BucketMover "Will attempt to freeze" | rex field=_raw "(?P[^\\]+)\\(db|colddb)\\db(?P[\d]+)(?P[\d]+)(?P[\d]+)' (?P.*)" | convert ctime(earliest_epoch) as Earliest_Data | convert ctime(latest_epoch) as Latest_Data | convert ctime(_time) as Log_TimeStamp | table Log_TimeStamp,index_name,bucket_number,Earliest_Data,Latest_Data,reason | sort - Log_TimeStamp

[glitchcowboy]

Had to modify rex for my db's from /logs/splunk_logs to /mydatadir/\w+

Extraordinary, thanks!