topic Optimize Nested Search in Splunk Search

Optimize Nested Search

antb — Thu, 08 Aug 2019 01:22:05 GMT

This search is slow (our dns logs are large).

index=winlogs sourcetype=dns | eval dottedquestion=replace(replace(questionname,"\(\d+\)","."),"\.(.*)\.","\1") | search [| inputlookup baddomains | return 10000 dottedquestion=Domain]

Outside of shrinking the time window (I am not interested in going under 24 hours) is there anyway to optimize it? The baddomains list is very small (<1000)

Thank you in advance.

Re: Optimize Nested Search

jawaharas — Thu, 08 Aug 2019 06:43:09 GMT

Can you share some sample pattern for questionname field?

Re: Optimize Nested Search

antb — Thu, 08 Aug 2019 11:17:07 GMT

Sure - apparently ms logs dns in “pascal style” string format. Showing the length of each next section in parens ending in (0).

(12)somecomputer(6)domain(3)com(0)

Defined in 4.1.2 of the RFC1035:
https://tools.ietf.org/html/rfc1035

Re: Optimize Nested Search

jpolvino — Thu, 08 Aug 2019 12:23:04 GMT

If you're not counting and just looking for presence, try a "dedup dottedquestion" just before the | search.

Re: Optimize Nested Search

jawaharas — Fri, 09 Aug 2019 05:12:18 GMT

Probably join might help you.

index=winlogs sourcetype=dns 
| eval Domain=replace(replace(questionname,"\(\d+\)","."),"\.(.*)\.","\1") 
| join type=inner Domain 
    [| inputlookup baddomains 
    | table Domain]

Also, if it's ad-hoc search, Run in 'Fast Mode' instead of 'Verbose Mode'.