Deployment Architecture

My warm bucket moves straight to frozen and I don't know why?

New Member

This is my indexes.conf:
path = /data/hot
maxVolumeDataSizeMB = 8500
path = /data/cold
maxVolumeDataSizeMB = 10500
homePath = volume:hot/myindex/db
coldPath = volume:cold/myindex/colddb
thawedPath = /data/cold/myindex/thaweddb
coldToFrozenDir = /data/cold/myindex/frozendb

hot to warm

maxDataSize = 20

2 hours to move from hot to warm

maxHotSpanSecs = 7200

100mb for cold to fill up

coldPath.maxDataSizeMB = 100
maxTotalDataSizeMB = 150

maximum warm buckets

maxWarmDBCount = 2
frozenTimePeriodInSecs = 1800

I uploaded a 2mb zip which extracts to 18mb of files onto the webUI. It creates a hot bucket in the hot/warm db folder. I upload the same zip again onto the webUI which then converts the hot bucket to a warm bucket due to maxDataSize = 20 with the remainder going into a new hot bucket. After a minute or so the warm bucket moves directly to frozen without going to cold. Also, maxHotSpanSecs = 7200 doesn't seem to work either ie my hot buckets don't roll to warm after 7200 seconds (2 hours) Could anyone tell me what is wrong with my indexes.conf file?
Thank you!

0 Karma

Path Finder

To add to teunlaan's answer - if all events in your bucket are older than 1800 seconds it will freeze. You can check buckets using this search:

|dbinspect index=YOUR_INDEX_NAME
|eval span=replace(replace(replace(replace(tostring(ceiling(endEpoch-startEpoch), "duration"),"(?:(\d+)\+?)(\d+)\:(\d+):(\d+)","\1d \2h \3m \4s"), "(?<!\d)0+[dhms]", ""), "^\s+$", "00s"), "0(?=\d[dhms])", "")
|convert ctime(startEpoch) as startDate, ctime(endEpoch) as endDate
|table splunk_server, index, state, startDate, endDate, span, sizeOnDiskMB

To answer your other question about maxHotSpanSecs not working, it's because you hit the maxDataSize first so the hot rolls to warm based on size. The first condition to hit (either time or size) wins.

0 Karma


frozenTimePeriodInSecs = 1800

What is the Timestamp of the logs?

If they are older than 1800 seconds, (if they alre the only event in the bucket)

0 Karma
Get Updates on the Splunk Community!

Introducing Edge Processor: Next Gen Data Transformation

We get it - not only can it take a lot of time, money and resources to get data into Splunk, but it also takes ...

Take the 2021 Splunk Career Survey for $50 in Amazon Cash

Help us learn about how Splunk has impacted your career by taking the 2021 Splunk Career Survey. Last year’s ...

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...