Monitoring Splunk

What would be the best bucket rotation policy to enhance Splunk performance with our current index file configuration?

Contributor

We have license of 100GB of data for indexing per day. Client requirement is to have 60 days of searchable data after which data can be moved to frozen. below is the index file we configure. could u plz suggest the best bucket rotation policy which would enhance splunk performance.

2x six-core, 2+ GHz CPU, 12 GB RAM, RAID 0 or 1+0, with a 64 bit OS installed.

[test_index]
homePath = $SPLUNK_DB/testdb/db
coldPath = $SPLUNK_DB/testdb/colddb
thawedPath = $SPLUNK_DB/testdb/thaweddb
maxDataSize = 3072000
maxHotBuckets = 1
maxWarmDBCount = 2
maxTotalDataSizeMB = 6144000
frozenTimePeriodInSecs = 5184000
coldToFrozenDir = "$SPLUNK_HOME/myfrozenarchive"
0 Karma
1 Solution

Splunk Employee
Splunk Employee

you do not want maxHotBuckets = 1, it will cause too many buckets to be rolled every time an event comes from a different range, you want at least 3, up to 10.

In addition to the maxDataSize, you may want to add a limit on the timerange per bucket (maybe several days) to avoid too wide buckets and fasten the deletion once reached the frozenTimePeriodInSecs (see maxHotSpanSecs) (PS NEVER NEVER use 3600 or 86400 as values)

http://docs.splunk.com/Documentation/Splunk/6.2.0/admin/Indexesconf

View solution in original post

Splunk Employee
Splunk Employee

you do not want maxHotBuckets = 1, it will cause too many buckets to be rolled every time an event comes from a different range, you want at least 3, up to 10.

In addition to the maxDataSize, you may want to add a limit on the timerange per bucket (maybe several days) to avoid too wide buckets and fasten the deletion once reached the frozenTimePeriodInSecs (see maxHotSpanSecs) (PS NEVER NEVER use 3600 or 86400 as values)

http://docs.splunk.com/Documentation/Splunk/6.2.0/admin/Indexesconf

View solution in original post

State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!