So i have been looking for ways to improve performance of my dashboards - to give you summary - i currently have 3 Dashboards and each dashboard is running more than 50 Searches at a given time, all the dashboards are hosted on 1 Search head which is feed by 2 indexers for data.
i have been reading and seems like Summary Indexing along with Schedule search is solution i am looking but i have few doubts and would greatly appreciate if some on can chime in 2 cents based on past experience:
1. I am planning to create 150 Scheduled Searches (none of them is duplicate) to run every 15 Minutes ( i want to run them every 15 minutes because - we need real time dashboard)
2. I will then put output of these schedule searches in different summary indexes (because of different people need access to different data)
3. Then run my Advanced XML dashboards against these Summary index saved results.
Questions i have:
1. Is this right move? or is there is anything better which can help improve dashboard performance. (at any given time we can have up-to 20 people logged in and looking at real time dashboards).
2. One thing i noted is that Summary Indexes get data from Scheduled Searches which atleast 1 Hour behind in time, why is it so? if my Adv XML is running a search against Summary Index then that means i can never get data which is near real time ? i do not want to wait for 1 hr before data for now shows up on dashboard - am i missing something?
You're on the rigth way but you forgoted something or/and you make some mistakes.
Summary indexes are there to accelerate the results of searches but it's not the only thing.
You must know that the first condition to respect so that the performance of your dashboards can be improved is that each of your dashboards most not have more than 8 searches. This condition is essential for splunk to displays and runs correctly your dashboards.
So before using Summary indexes, make sure that your dashboards respect that condition.