Deployment Architecture

Estimating volume requirements for internal indexes?

rturk
Builder

Hi Splunkers!

So as part two of my storage estimation (part one HERE), I have to allow for the growth & storage of the internal indexes in the following environment:
- 100GB/day
- 90 days data retention

Using the rough estimation of daily volume x retention x 1/2 I get 4.5TB of storage required for the above specs.

What kind of percentage buffer should I be applying for my Splunk Internal indexes (i.e. _internal, _audit, _fishbucket, etc..). I vaguely recall seeing a value of 15% being mentioned somewhere but that may have applied to something else.

Thanks in advance 🙂

Tags (3)
0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

They are limited by default to 500GB each, and have a short retention of 30 days.
You can size them as needed.
However the fishbucket is not a classic index, it contains the trace of the files locally monitored and can grow (usually on forwarders)

[EDIT}

the actual defaults are in $SPLUNK_HOME/etc/system/default/indexes.conf
maxTotalDataSizeMB = 500000

View solution in original post

yannK
Splunk Employee
Splunk Employee

They are limited by default to 500GB each, and have a short retention of 30 days.
You can size them as needed.
However the fishbucket is not a classic index, it contains the trace of the files locally monitored and can grow (usually on forwarders)

[EDIT}

the actual defaults are in $SPLUNK_HOME/etc/system/default/indexes.conf
maxTotalDataSizeMB = 500000

rturk
Builder

Thanks yannK (and sorry for the delay!)

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...