Deployment Architecture

My warm bucket moves straight to frozen and I don't know why?

Chrisdarn
New Member

This is my indexes.conf:
*[volume:hot]
path = /data/hot
maxVolumeDataSizeMB = 8500
[volume:cold]
path = /data/cold
maxVolumeDataSizeMB = 10500
[myindex]
homePath = volume:hot/myindex/db
coldPath = volume:cold/myindex/colddb
thawedPath = /data/cold/myindex/thaweddb
coldToFrozenDir = /data/cold/myindex/frozendb

hot to warm

maxDataSize = 20

2 hours to move from hot to warm

maxHotSpanSecs = 7200

100mb for cold to fill up

coldPath.maxDataSizeMB = 100
maxTotalDataSizeMB = 150

maximum warm buckets

maxWarmDBCount = 2
frozenTimePeriodInSecs = 1800
*

I uploaded a 2mb zip which extracts to 18mb of files onto the webUI. It creates a hot bucket in the hot/warm db folder. I upload the same zip again onto the webUI which then converts the hot bucket to a warm bucket due to maxDataSize = 20 with the remainder going into a new hot bucket. After a minute or so the warm bucket moves directly to frozen without going to cold. Also, maxHotSpanSecs = 7200 doesn't seem to work either ie my hot buckets don't roll to warm after 7200 seconds (2 hours) Could anyone tell me what is wrong with my indexes.conf file?
Thank you!

0 Karma

kellewic
Path Finder

To add to teunlaan's answer - if all events in your bucket are older than 1800 seconds it will freeze. You can check buckets using this search:

|dbinspect index=YOUR_INDEX_NAME
|eval span=replace(replace(replace(replace(tostring(ceiling(endEpoch-startEpoch), "duration"),"(?:(\d+)\+?)(\d+)\:(\d+):(\d+)","\1d \2h \3m \4s"), "(?<!\d)0+[dhms]", ""), "^\s+$", "00s"), "0(?=\d[dhms])", "")
|convert ctime(startEpoch) as startDate, ctime(endEpoch) as endDate
|table splunk_server, index, state, startDate, endDate, span, sizeOnDiskMB

To answer your other question about maxHotSpanSecs not working, it's because you hit the maxDataSize first so the hot rolls to warm based on size. The first condition to hit (either time or size) wins.

0 Karma

teunlaan
Contributor

frozenTimePeriodInSecs = 1800

What is the Timestamp of the logs?

If they are older than 1800 seconds, (if they alre the only event in the bucket)

0 Karma
Get Updates on the Splunk Community!

Splunk APM & RUM | Upcoming Planned Maintenance

There will be planned maintenance of the streaming infrastructure for Splunk APM and Splunk RUM in the coming ...

Part 2: Diving Deeper With AIOps

Getting the Most Out of Event Correlation and Alert Storm Detection in Splunk IT Service Intelligence   Watch ...

User Groups | Upcoming Events!

If by chance you weren't already aware, the Splunk Community is host to numerous User Groups, organized ...