Monitoring Splunk

What would be the best bucket rotation policy to enhance Splunk performance with our current index file configuration?

Venkat_16
Contributor

We have license of 100GB of data for indexing per day. Client requirement is to have 60 days of searchable data after which data can be moved to frozen. below is the index file we configure. could u plz suggest the best bucket rotation policy which would enhance splunk performance.

2x six-core, 2+ GHz CPU, 12 GB RAM, RAID 0 or 1+0, with a 64 bit OS installed.

[test_index]
homePath = $SPLUNK_DB/testdb/db
coldPath = $SPLUNK_DB/testdb/colddb
thawedPath = $SPLUNK_DB/testdb/thaweddb
maxDataSize = 3072000
maxHotBuckets = 1
maxWarmDBCount = 2
maxTotalDataSizeMB = 6144000
frozenTimePeriodInSecs = 5184000
coldToFrozenDir = "$SPLUNK_HOME/myfrozenarchive"
0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

you do not want maxHotBuckets = 1, it will cause too many buckets to be rolled every time an event comes from a different range, you want at least 3, up to 10.

In addition to the maxDataSize, you may want to add a limit on the timerange per bucket (maybe several days) to avoid too wide buckets and fasten the deletion once reached the frozenTimePeriodInSecs (see maxHotSpanSecs) (PS NEVER NEVER use 3600 or 86400 as values)

http://docs.splunk.com/Documentation/Splunk/6.2.0/admin/Indexesconf

View solution in original post

yannK
Splunk Employee
Splunk Employee

you do not want maxHotBuckets = 1, it will cause too many buckets to be rolled every time an event comes from a different range, you want at least 3, up to 10.

In addition to the maxDataSize, you may want to add a limit on the timerange per bucket (maybe several days) to avoid too wide buckets and fasten the deletion once reached the frozenTimePeriodInSecs (see maxHotSpanSecs) (PS NEVER NEVER use 3600 or 86400 as values)

http://docs.splunk.com/Documentation/Splunk/6.2.0/admin/Indexesconf

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...