Deployment Architecture
Highlighted

Estimating volume requirements for internal indexes?

Builder

Hi Splunkers!

So as part two of my storage estimation (part one HERE), I have to allow for the growth & storage of the internal indexes in the following environment:
- 100GB/day
- 90 days data retention

Using the rough estimation of daily volume x retention x 1/2 I get 4.5TB of storage required for the above specs.

What kind of percentage buffer should I be applying for my Splunk Internal indexes (i.e. _internal, _audit, _fishbucket, etc..). I vaguely recall seeing a value of 15% being mentioned somewhere but that may have applied to something else.

Thanks in advance 🙂

Tags (3)
0 Karma
Highlighted

Re: Estimating volume requirements for internal indexes?

Splunk Employee
Splunk Employee

They are limited by default to 500GB each, and have a short retention of 30 days.
You can size them as needed.
However the fishbucket is not a classic index, it contains the trace of the files locally monitored and can grow (usually on forwarders)

[EDIT}

the actual defaults are in $SPLUNK_HOME/etc/system/default/indexes.conf
maxTotalDataSizeMB = 500000

View solution in original post

Highlighted

Re: Estimating volume requirements for internal indexes?

Builder

Thanks yannK (and sorry for the delay!)