topic Re: Why is the dropdown selection with large dataset, slow? in Dashboards & Visualizations

Why is the dropdown selection with large dataset, slow?

koshyk — Fri, 13 Dec 2019 10:52:50 GMT

hi,
we have few dashboards for end users to select "hostname" to search for. Unfortunately, the list of unique hostnames now renders above 25K+ and the Splunk default Dropdown element (with typing option) is slow

Any ready-made code you guys have found to load a large dataset dynamically? Something like when I type prod... , it should dynamically load the servers with prod* in its name and not the entire set of 25k+ hostnames beforehand.

I could fiddle out with select2 library which have similar functions or some external libraries UI work, but was checking if you guys have already done so within Splunk native UI elements.

Re: Why is the dropdown selection with large dataset, slow?

gcusello — Fri, 13 Dec 2019 11:49:47 GMT

Hi @koshyk,
when I have a so large set of values I don't use dropdown but text box!

If you need a dropdown and you haven't a so dynamic list, you could try to extract a lookup with a scheduled search and use it for your dropdown.

Ciao.
Giuseppe

Re: Why is the dropdown selection with large dataset, slow?

jpolvino — Fri, 13 Dec 2019 13:07:49 GMT

I don't have host lists that big, but like gcusello would avoid drop down lists for this. One solution that sounds like what you're suggesting would be to have a pre-filter text box that limits the list to "prod*" when you type prod into it. If your hosts belong to families (e.g., prod, nonprod, test, perf), then a drop down with hose values can help limit the host list. You'd want to put the user input before the first pipe and also limit the dynamic search time range to something reasonable. You can also explore using a summary index to collect unique host names every day (not the typical stats use case SI is famous for) and then query that. Of course, put pre-pipe fields in the lookup to tighten your search: index, sourcetype, earliest, latest, source, etc.

Re: Why is the dropdown selection with large dataset, slow?

tomaszpiekos — Fri, 13 Dec 2019 14:43:54 GMT

maybe like this
|tstats values(host) where index=* by host | fields host | dedup host

restrict time range to the reasonable minimum.

Re: Why is the dropdown selection with large dataset, slow?

koshyk — Tue, 21 Jan 2020 11:24:11 GMT

I understand from technical point of view. But the requirement is from client which I have less flexiblity to change upon. the reason to have drop down is to fill up while users type as they don't to make mistakes with incorrect names

Re: Why is the dropdown selection with large dataset, slow?

to4kawa — Tue, 21 Jan 2020 11:33:06 GMT

textbox-validation-issue

why don't you lookup host and check host is?