Splunk Search

Configure timespan bucket

wvalente
Explorer

Hi guys,

I have to configure the timespan to roll data to warm, cold and frozen.

The question is:

How can configure timespan to roll from hot to warm?

And from warm to cold? And from cold to frozen?

This configurations must be done in each index in indexes.conf?

I read the http://docs.splunk.com/Documentation/Splunk/6.5.2/Indexer/Configureindexstorage and http://docs.splunk.com/Documentation/Splunk/6.5.2/admin/Indexesconf document but I only found frozenTimePeriodInSecs not like "hotTimePeriodInSecs"

Thank you so much guys!

Tags (1)
0 Karma

somesoni2
Revered Legend

This is how the data buckets roll in Splunk

first stage: hot bucket
Data is actively written and searched in hot buckets. There can be maxHotBuckets number hot buckets written at a time. When one of following condition happens, hot bucket rolls to warm bucket
1) splunkd restart
2) bucket reaches size defined by maxDataSize
3) bucket reaches age defined by maxHotSpanSecs

second stage: warm bucket
Data is NOT written but actively searched. There can be maxWarmDBCount in the homePath directory. They roll to cold if
1) maxWarmDBCount exceeds
2) homePath/volume size limit exceeds

third stage: cold
Read-only and considered not actively searched. This rolls to frozen if age of all events in the cold bucket exceeds frozenTimePeriodInSecs or total index size exceeds maxTotalDataSizeMB.

wvalente
Explorer

Hi somesoni2

Tks for your response.

I understand. But I have to configure maxHotSpanSecs in which sector of the indexes.conf? In each index stanza or in "index specific defaults"

Other point: warm bucket to cold bucket do I have maxWarmSpanSecs too?

Thanks again ma friend.

0 Karma

ddrillic
Ultra Champion

Just be careful with maxHotSpanSecs with its default of 7776000 - 90 days! one major objective is to have fewer buckets as possible and this value helps in this regard. If you reduce it and the flow of data into this index is relatively low, you can create lots of small buckets, that the OS might not like.

xpac
SplunkTrust
SplunkTrust

You can put in default, which would make it valid for all index that do not have a different value configured in their own stanza.

There is no maxWarmSpanSecs at all. They on roll when one of the above mentioned criterias is fulfilled.

0 Karma
Get Updates on the Splunk Community!

New Dates, New City: Save the Date for .conf25!

Wake up, babe! New .conf25 dates AND location just dropped!! That's right, this year, .conf25 is taking place ...

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...