Monitoring Splunk

If maxTotalDataSizeMB is reached, how much data will Splunk remove from the index and will it freeze or delete it?

jonathanfalconi
Explorer

Hi,

If the maxTotalDataSizeMB is reached, how much data will splunk remove from the index and will it freeze it or simply delete?

Lastly, what can you search for in the splunkd logs to indicate the max size was reached?

thanks.

1 Solution

lguinn2
Legend

If the max size is reached, Splunk will freeze the oldest bucket in the index to make room for the new data. if you have not specified a cold-to-frozen directory or script, then Splunk will delete the oldest bucket instead. Splunk will only remove sufficient buckets to reduce the index size beneath the maximum, so that is probably just one bucket - but it may repeat this action frequently as new data arrives.

I played around with this a bit, and I think this will tell you what's happening as buckets are frozen:

index=_internal sourcetype=splunkd yourindexname component=BucketMover

View solution in original post

somesoni2
Revered Legend

This is a good read on Splunk data retention policy implementation.

http://wiki.splunk.com/Deploy:BucketRotationAndRetention

lguinn2
Legend

If the max size is reached, Splunk will freeze the oldest bucket in the index to make room for the new data. if you have not specified a cold-to-frozen directory or script, then Splunk will delete the oldest bucket instead. Splunk will only remove sufficient buckets to reduce the index size beneath the maximum, so that is probably just one bucket - but it may repeat this action frequently as new data arrives.

I played around with this a bit, and I think this will tell you what's happening as buckets are frozen:

index=_internal sourcetype=splunkd yourindexname component=BucketMover

jonathanfalconi
Explorer

grep'ing splunkd.log for 'bucketmover' I was able to find when and which buckets were frozen - thank you.

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...