I have 50+ million lines forwarded to an index.
It is in csv format and here is an example:
timestamp,teamname,buildnumber,url,latency,responsecode, (and few other fields)
Now, i am creating a dashboard with 2 drop-downs at the top.
First drop-down is unique teamname
Second drop-down is unique buildnumber based on the teamname (first drop-down)
The issue i have is that the first drop-down takes minutes to load.
Is there a way to make this any faster?
Is it possible to cache this ?
or using a lookup table an option here?
I am on version 6.2.1
Thanks
You could schedule a saved search which outputs the teamnames to csv and load those into your dropdown (http://docs.splunk.com/Documentation/Splunk/6.2.2/SearchReference/Outputcsv and http://docs.splunk.com/Documentation/Splunk/6.2.2/SearchReference/inputlookup).
Depending on how fresh your data has to be, you could run this search once per night, so your dropdown has results that are up to date each day. If you need more recent (i.e., hourly) data, you could also do a second search over a timeframe of the last 24 hours which starts on loading the dashboard and conditionally add any new results from that search to your dropdown dynamically.
But if I understood your sitation correctly, you have a large data basis with many "historic" events, so it may be more efficient to do a search over all your data once to get a basis for the teamnames and then only search once per day for any new entries which could then be added to your list teamnames. I am not sure if you need to do this by hand (e.g. create a csv file), or if this is somehow inbuilt with accelerated splunk reports.