Splunk Search

How do I get the difference in results from two searches over different timelines?

ghemanth
New Member

I want to get top 20 errors of the day & top 20 errors of the week. Then, I want to get the difference between both results. i.e. new errors that were seen in last 24 hrs which were not seen earlier.

I tried this, but it throws some error:

| multisearch [search ERROR earliest=-1d  | top limit=20 error_field | eval type="search1" ] [search ERROR earliest=-8d latest=-2d | top limit=20 error_field | eval type="search2"] | eval difference = search1-search2

Error thrown:

Multisearch subsearches may only contain purely streaming operations (subsearch 1 contains a non-streaming command.)
0 Karma
1 Solution

BonMot
Explorer

Mutisearch isn't doing what you think. It is literally running both searches in parallel and returning the results interwoven as they are found. Since you have to wait for both searches to complete, you may as well append and count up the results Here is a version I did to compare security alerts today vs last 7 days. The first search time picker is "Today"

ERROR 
| top limit=20 error_field
| eval type="day" 
| append 
    [ search ERROR earliest=-7d 
        | top limit=20 error_field 
        | eval type="week"
    ] 
| stats count,values(type) AS type by error_field  
|  where count=1 and type="day"

What is going on here is we're labeling each search with the eval command, then getting the top error_field and type for each search and concatenating everything into one result. Finally, we run some stats to eliminate any error_fields in both search sets and what's left is the error_field and type of the unique entries of type "day".

NOTE: Edited to incorporate improvements from @ghemanth's answer

View solution in original post

ghemanth
New Member

Thanks @BonMot.

It worked for me after re-arranging the 'eval' and 'top' paramters

 ERROR 
 | top limit=20 error_field,type 
 | eval type="day" 
 | append 
     [ search ERROR earliest=-7d 
         | top limit=20 error_field ,type 
         | eval type="week" 
     ] 
 | stats count,values(type) AS type by error_field  
 |  where count=1 and type="day"
0 Karma

BonMot
Explorer

That is actually cleaner. In that case you don't need to have type in your top command. I'll update my answer

0 Karma

BonMot
Explorer

Mutisearch isn't doing what you think. It is literally running both searches in parallel and returning the results interwoven as they are found. Since you have to wait for both searches to complete, you may as well append and count up the results Here is a version I did to compare security alerts today vs last 7 days. The first search time picker is "Today"

ERROR 
| top limit=20 error_field
| eval type="day" 
| append 
    [ search ERROR earliest=-7d 
        | top limit=20 error_field 
        | eval type="week"
    ] 
| stats count,values(type) AS type by error_field  
|  where count=1 and type="day"

What is going on here is we're labeling each search with the eval command, then getting the top error_field and type for each search and concatenating everything into one result. Finally, we run some stats to eliminate any error_fields in both search sets and what's left is the error_field and type of the unique entries of type "day".

NOTE: Edited to incorporate improvements from @ghemanth's answer

Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...