Deployment Architecture

Estimating volume requirements for internal indexes?

rturk
Builder

Hi Splunkers!

So as part two of my storage estimation (part one HERE), I have to allow for the growth & storage of the internal indexes in the following environment:
- 100GB/day
- 90 days data retention

Using the rough estimation of daily volume x retention x 1/2 I get 4.5TB of storage required for the above specs.

What kind of percentage buffer should I be applying for my Splunk Internal indexes (i.e. _internal, _audit, _fishbucket, etc..). I vaguely recall seeing a value of 15% being mentioned somewhere but that may have applied to something else.

Thanks in advance 🙂

Tags (3)
0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

They are limited by default to 500GB each, and have a short retention of 30 days.
You can size them as needed.
However the fishbucket is not a classic index, it contains the trace of the files locally monitored and can grow (usually on forwarders)

[EDIT}

the actual defaults are in $SPLUNK_HOME/etc/system/default/indexes.conf
maxTotalDataSizeMB = 500000

View solution in original post

yannK
Splunk Employee
Splunk Employee

They are limited by default to 500GB each, and have a short retention of 30 days.
You can size them as needed.
However the fishbucket is not a classic index, it contains the trace of the files locally monitored and can grow (usually on forwarders)

[EDIT}

the actual defaults are in $SPLUNK_HOME/etc/system/default/indexes.conf
maxTotalDataSizeMB = 500000

rturk
Builder

Thanks yannK (and sorry for the delay!)

Get Updates on the Splunk Community!

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Answers Content Calendar, July Edition I

Hello Community! Welcome to another month of Community Content Calendar series! For the month of July, we will ...