Dashboards & Visualizations
Highlighted

How to pre-query data results to avoid constantly searching

Explorer

I've got 4 dropdowns(Country,Year,Month,Day) getting information from my index and those will then populate some charts.

This not only takes some time as it is doing 4 queries simultaneously which can be degrading for the server's performance (if several people do this it is x4 queries per person)

One idea i've had was to pre-query the information, and that would run based of that. That would avoid not only the waiting time but also impact on server performance.

How would i go about this?
Create searches based on each of the dropdowns and put on each dropdown the "load search"?

Best Regards,
Rui

0 Karma
Highlighted

Re: How to pre-query data results to avoid constantly searching

Communicator

Seems too simple, but could you populate the 4 drop downs manually in the panel so that Splunk wouldn't need to create them itself?

0 Karma
Highlighted

Re: How to pre-query data results to avoid constantly searching

Explorer

Well, it could but the problem is that the options can change anytime(such as days and months available on query).
That was the main reason why i turned it dynamic

0 Karma
Highlighted

Re: How to pre-query data results to avoid constantly searching

Communicator

Well, have you tried using something like this?

Highlighted

Re: How to pre-query data results to avoid constantly searching

Explorer

Ok i'll take a look into it! Thanks a lot for your input on this.

0 Karma
Highlighted

Re: How to pre-query data results to avoid constantly searching

Esteemed Legend

First create a search that generates your stuff, one search for each control. Then save and schedule this search to run every [whatever]. In your control, you can use a populating search that uses |loadjob to pull in the results from the most recent run of your search. Or, you can have the search end in |outputlookup or |outputcsv and you can pull in the data with |inputlookup or |inputcsv, but this is an extra step that obfuscates the true source of the data (which is your search).

Highlighted

Re: How to pre-query data results to avoid constantly searching

SplunkTrust
SplunkTrust

I'm not sure why you would have four searches to get Country, Year, Month and Day to populate dropdowns. That's really only two fields, and how you break them up. I'm also not sure why that search would be particularly time consuming. If Country is an indexed field, and the relevant Date field is an indexed field, then you can have an initial search using tstats to create records with Country and Date, then use post-processing to split out the pieces

 | tstats count as count0 WHERE index=foo BY Country span=1d

This should quickly give you a record for each Country for each Date there is data for that Country, assuming the event's _time contains the Date value in question. On the other hand, if the required Date is in an indexed field called whateverDate, you would do something like this.

 | tstats count as count0 WHERE index=foo BY Country, whateverDate