Splunk Search

Best Methods to Improve Performance of Dashboard

katzr
Path Finder

I have a dashboard with ~38 panels with 2 joins per panel. I'm curious what is the most costly for Splunk performance of a dashboard- is it the large number of panels I have or is it the number of joins I have in each?

What are some common ways to improve the performance of a dashboard? Below is an example of one of my panels. I am doing some weird things with my location info because using the default value setting in my lookup table was throwing me a weird error.

 index=example date_month=August date_year=2017 (assignment_group="*") | dedup number | fillnull value="UNKNOWN" location | eval regionblank= "UNKNOWN" | eval countryblank= "UNKNOWN" | eval locationblank="UNKNOWN" | lookup CurrentSiteInfo.csv location| eval site=coalesce(location2,locationblank) | eval Region=coalesce(Region,regionblank)| eval Country=coalesce(Country,countryblank) | search ((Region="*") (Country="*") (site="*")) |stats count as Tickets by contact_type | join overwrite=false contact_type [search index=example earliest="6/01/2017:00:00:00" latest="12/31/2017:24:00:00" (assignment_group="*") | dedup number | fillnull value="UNKNOWN" location | eval regionblank= "UNKNOWN" | eval countryblank= "UNKNOWN" | eval locationblank="UNKNOWN" | lookup CurrentSiteInfo.csv location| eval site=coalesce(location2,locationblank) | eval Region=coalesce(Region,regionblank)| eval Country=coalesce(Country,countryblank) | search ((Region="*") (Country="*") (site="*"))| bucket _time span=1mon | stats count as Tickets by contact_type _time | stats avg(Tickets) as Baseline by contact_type | eval Baseline = round(Baseline,0)] | eval "Baseline Variance" = Tickets - Baseline | join overwrite=false contact_type [search index=example earliest=-3mon@mon (assignment_group="*") | dedup number | fillnull value="UNKNOWN" location | eval regionblank= "UNKNOWN" | eval countryblank= "UNKNOWN" | eval locationblank="UNKNOWN" | lookup CurrentSiteInfo.csv location| eval site=coalesce(location2,locationblank) | eval Region=coalesce(Region,regionblank)| eval Country=coalesce(Country,countryblank) | search ((Region="*") (Country="*") (site="*")) | bucket _time span=1mon | stats count as Tickets by contact_type _time | stats avg(Tickets) as Average by contact_type | eval Average = round(Average,0)] | eval "Average Variance" = Tickets - Average | table contact_type Tickets Baseline "Baseline Variance" Average "Average Variance" | addcoltotals | sort 0 Tickets
0 Karma

DalJeanis
Legend

Okay, I know we've already reviewed and simplified that search for you.

Ah, you commented on it an hour ago. We've given you that code over there.

https://answers.splunk.com/answers/578622/how-to-make-my-search-more-efficient-help-to-remov.html

Now, if you'd let us know how the different searches are different from each other, then we can discuss how to set up a base search + postprocessing in order to meet your need.

0 Karma

katzr
Path Finder

Hi @DalJeanis I appreciate all of your help. A lot of the searches are the same format as the one above, but are looking at it from tickets by region, problem, category, etc. instead of contact_type. But the baseline and average parameters are the same.

0 Karma

adonio
Ultra Champion

ask yourself if 38 panels are really necessary.
you are tying 38 cores when you are starting this dashboard...
there are plenty of other tricks like base search and more

0 Karma

MuS
Legend

Additionally to this advice, use a base search and post process the results http://docs.splunk.com/Documentation/Splunk/latest/Viz/Savedsearches#Post-process_searches_2
As base search use a search that skips the join and use stats instead, read here http://sideviewapps.com/slides/2017_05_02_sideview_let_stats_sort_them_out.pptx to get some create advice on how to use stats or here https://answers.splunk.com/answers/129424/how-to-compare-fields-over-multiple-sourcetypes-without-jo...

cheers, MuS

0 Karma

katzr
Path Finder

This dashboard is for a customer who requested this high number of panels. Can you expand upon the "plenty of other tricks"

0 Karma

adonio
Ultra Champion

so really depends on the use cases and on what the panels cover.
you can create base searches, improve searches, set panels loading order look here: https://answers.splunk.com/answers/513660/how-to-set-loading-order-for-panels.html
regardless, 38 panels is plenty, try and sort out the use cases and maybe split this dashboard to 5-6 dashboards.
you can always increase the cpu on the system if this is a possibility
hope it helps

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...