Splunk Search

Custom Index

mavilla
Explorer

I have one specific dashboard that I monitor with 7 tiles on it, there are times when the dashboard searches auto cancel and it is slow to load as well, once it is reloaded the results of the searched come back for a period of time then auto cancel again. Currently there is one main index for everything that's coming into Splunk, would setting up a separate index just for the searched in this specific dashboard fix some of these errors? All the searches in this dashboard need to be real time.

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

First, it's rarely a good idea to put all data into a single index.
Second, putting the data into separate indexes probably won't help as the limiting factor is number of search slots (CPUs) available at the moment.
Third, a dashboard does NOT need to be real-time. Real-time searches should be used when the response will be real-time. For people watching a dashboard, a 1-minute or 5-minute interval is sufficient. By using real-time searches, your dashboard is tying up 7 CPUs on the search head and all indexers.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

First, it's rarely a good idea to put all data into a single index.
Second, putting the data into separate indexes probably won't help as the limiting factor is number of search slots (CPUs) available at the moment.
Third, a dashboard does NOT need to be real-time. Real-time searches should be used when the response will be real-time. For people watching a dashboard, a 1-minute or 5-minute interval is sufficient. By using real-time searches, your dashboard is tying up 7 CPUs on the search head and all indexers.

---
If this reply helps you, Karma would be appreciated.

mavilla
Explorer

Thank you for your answer Rich - just so I understand this correctly, you're saying that since I have 7 tiles on this dashboard and each of them is searching in real time, each tile is taking up one CPU to perform the searches? For context, our Splunk server is running 16 CPU cores (virtual), I see your point about the entire dashboard being in real time.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Yes. Seven real-time tiles/searches take up 7 CPUs, not only on the search head where the dashboard runs, but also on all indexers.

---
If this reply helps you, Karma would be appreciated.
0 Karma

mavilla
Explorer

Our search head and indexer are hosted on the same server, we've already identified this as being an issue but didn't know it was this much of an issue. Thanks again

0 Karma
Get Updates on the Splunk Community!

Using Machine Learning for Hunting Security Threats

REGISTER NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more ...

Security Highlights | November 2022 Newsletter

 November 2022 2022 Gartner Magic Quadrant for SIEM: Splunk Named a Leader for the 9th Year in a RowSplunk is ...

Platform Highlights | November 2022 Newsletter

 November 2022 Skill Up on Splunk with our New Builder Tech Talk SeriesCan you build it? Yes you can! *play ...