[Advice/Help] Lookup Table search taking long

[Player01]

Hello All,

Can ya'll give me advice on why my query taking so long? In a dashboard it just times out and regular verbose it takes quite a bit of time. Purpose of the query is to simply just search my index and output me the result that match in the lookup url. 

index=myindex sourcetype=mysource
| stats count by url
| fields - count
| search [|inputlookup LCL_url.csv | fields url]
| sort url

Thank you

[gcusello]

Hi @Player01 ,

Are all your searches slow or only this one?

how many logs have you in your index and in your lookup?

which storage have? is it compliant with the Splunk requirement of at least 800 IOPS?

Ciao.

Giuseppe

[yuanliu · Accepted solution]

If the lookup is of any size and if the number of events from the index search is large, yes, this will take vary long because it is effectively performing

| search url = "foo.com" OR url = "bar.com" OR url = "barz" OR ...

Moving the subsearch to index search can save some time because you won't be streaming as many events:

index=myindex sourcetype=mysource
  [|inputlookup LCL_url.csv | fields url]
| stats count by url
| fields - count
| sort url

However, using a lookup file as subsearch may not be the best option to begin with.

[gcusello]

Hi @Player01 ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉