Monitoring Splunk

What would be the best bucket rotation policy to enhance Splunk performance with our current index file configuration?

Venkat_16
Contributor

We have license of 100GB of data for indexing per day. Client requirement is to have 60 days of searchable data after which data can be moved to frozen. below is the index file we configure. could u plz suggest the best bucket rotation policy which would enhance splunk performance.

2x six-core, 2+ GHz CPU, 12 GB RAM, RAID 0 or 1+0, with a 64 bit OS installed.

[test_index]
homePath = $SPLUNK_DB/testdb/db
coldPath = $SPLUNK_DB/testdb/colddb
thawedPath = $SPLUNK_DB/testdb/thaweddb
maxDataSize = 3072000
maxHotBuckets = 1
maxWarmDBCount = 2
maxTotalDataSizeMB = 6144000
frozenTimePeriodInSecs = 5184000
coldToFrozenDir = "$SPLUNK_HOME/myfrozenarchive"
0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

you do not want maxHotBuckets = 1, it will cause too many buckets to be rolled every time an event comes from a different range, you want at least 3, up to 10.

In addition to the maxDataSize, you may want to add a limit on the timerange per bucket (maybe several days) to avoid too wide buckets and fasten the deletion once reached the frozenTimePeriodInSecs (see maxHotSpanSecs) (PS NEVER NEVER use 3600 or 86400 as values)

http://docs.splunk.com/Documentation/Splunk/6.2.0/admin/Indexesconf

View solution in original post

yannK
Splunk Employee
Splunk Employee

you do not want maxHotBuckets = 1, it will cause too many buckets to be rolled every time an event comes from a different range, you want at least 3, up to 10.

In addition to the maxDataSize, you may want to add a limit on the timerange per bucket (maybe several days) to avoid too wide buckets and fasten the deletion once reached the frozenTimePeriodInSecs (see maxHotSpanSecs) (PS NEVER NEVER use 3600 or 86400 as values)

http://docs.splunk.com/Documentation/Splunk/6.2.0/admin/Indexesconf

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...