Capacity planning best practices for Splunk Enterprise?

[adukes_splunk]

I'm looking for resources to help plan my deployment. Does anyone have capacity planning best practices for Splunk Enterprise?

[adukes_splunk · Accepted solution]

The Splunk Product Best Practices team provided this response. Read more about How Crowdsourcing is Shaping the Future of Splunk Best Practices.

Take a load off!

Capacity planning with Splunk isn't so straightforward. Got slow indexing? Add indexers. Got slow searching? Add indexers! I bet you weren't expecting that answer.

How capacity planning helps you scale your deployment

Best practices for capacity planning are to size the environment to the near-max, not the average load (unless you want to be wrong half the time). The Monitoring Console, available to admin users, contains a set of dashboards to provide insight into your deployment’s indexing and search performance, licensing, and OS resource usage. Let’s focus on the resource usage dashboards since this relates directly to pure system load and can be useful for capacity planning.
For information about the Monitoring Console in Splunk Cloud, see monitor Splunk Cloud deployment health in the Splunk Cloud User Manual.

Things to know

Capacity planning is not easy so don’t hesitate to contact Splunk for guidance on complex deployments.The Splunk Enterprise Capacity Planning Manual contains a lot of detail about hardware capacity planning and how to scale your Splunk Enterprise deployment, so while it’s important to understand how to apply these strategies, also read about the Monitoring Console and familiarize yourself with this monitoring tool to view topology and performance information.

Review and consider the following items as you plan your deployment:

• Dimensions of a Splunk Enterprise deployment: Helps determine if you need one or more machines can handle the indexing and search load.
• Distribute indexing and searching: Explains how both user count and a rise in the amount of indexed data impact performance levels.
• Performance questionnaire: Helps determine when you should add more hardware resources
• Splunk Storage Sizing: Estimates the average daily amount of data to be ingested.

The Splunk First 90 Days Program does not offer guidance on deployment technologies or deployment sizing because there are too many options to consider. For more information about architecture design, review the sample topologies in the Splunk Validated Architectures white paper to find repeatable topologies you can align with.

Things to do

• Find highs and lows. Use the resource usage dashboards on the Monitoring Console to identify the times during the day your data load is at it's highest and lowest. Use those numbers to determine the total capacity for your deployment.
• Lighten the data load. What's the total and average indexing performance? Consult the resource usage dashboards to look for indexing pipelines bottlenecks.