Getting Data In

maxHotSpanSecs not rolling hot buckets

jek01
New Member

I use "maxHotSpanSecs" to cut the size of each bucket received.
Only join "maxHotSpanSecs = 2592000" (30d) in test of local/indexes.conf
(index=test)
Execution results: Each bucket is greater than 30 days . EX: one bucket :2016/04/01 ~2017/02/08
Do not know why the cutting is unsuccessful?
TKS.

0 Karma

jkat54
SplunkTrust
SplunkTrust

how long have you had these settings in place? You mention one bucket has data from 2016/04/01 ~2017/02/08.... how about all the buckets since you made this change?

Splunk will not go back in time and readjust buckets to your new boundaries. That is to say, if you didnt have these settings before, the buckets would have contained upwards of 10GB / 90 days whichever is greater. Also, fringe events can be indexed into the same bucket when they arrive out of order.

For example, if I have cold and warm buckets from 2015 & 2016, and a hot bucket for 2017... and events come in with timestamps from 2014... the 2014 events will be dropped into the hot bucket. Now this hot bucket will show it has data from 2014 - 2017. In new and large environments, this happens all the time as you on-board new data sources.

0 Karma

jek01
New Member

[test]
coldPath = $SPLUNK_DB/test/colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB/test/db
maxTotalDataSizeMB = 512000
thawedPath = $SPLUNK_DB/test/thaweddb
maxHotSpanSecs = 259200
maxWarmDBCount = 500

[test_1]
coldPath = $SPLUNK_DB/test_1/colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB/test_1/db
maxTotalDataSizeMB = 512000
thawedPath = $SPLUNK_DB/test_1/thaweddb
maxHotSpanSecs = 259200
maxHotBuckets = 1
maxWarmDBCount = 500

0 Karma

MuS
Legend

In the second stanza you set maxHotBuckets=1 which will do this:

 NOTE: If you set maxHotBuckets to 1, Splunk attempts to send all
events to the single hot bucket and `maxHotSpanSecs` will not be
enforced
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...