Splunk Search

How do I get the difference in results from two searches over different timelines?

ghemanth
New Member

I want to get top 20 errors of the day & top 20 errors of the week. Then, I want to get the difference between both results. i.e. new errors that were seen in last 24 hrs which were not seen earlier.

I tried this, but it throws some error:

| multisearch [search ERROR earliest=-1d  | top limit=20 error_field | eval type="search1" ] [search ERROR earliest=-8d latest=-2d | top limit=20 error_field | eval type="search2"] | eval difference = search1-search2

Error thrown:

Multisearch subsearches may only contain purely streaming operations (subsearch 1 contains a non-streaming command.)
0 Karma
1 Solution

BonMot
Explorer

Mutisearch isn't doing what you think. It is literally running both searches in parallel and returning the results interwoven as they are found. Since you have to wait for both searches to complete, you may as well append and count up the results Here is a version I did to compare security alerts today vs last 7 days. The first search time picker is "Today"

ERROR 
| top limit=20 error_field
| eval type="day" 
| append 
    [ search ERROR earliest=-7d 
        | top limit=20 error_field 
        | eval type="week"
    ] 
| stats count,values(type) AS type by error_field  
|  where count=1 and type="day"

What is going on here is we're labeling each search with the eval command, then getting the top error_field and type for each search and concatenating everything into one result. Finally, we run some stats to eliminate any error_fields in both search sets and what's left is the error_field and type of the unique entries of type "day".

NOTE: Edited to incorporate improvements from @ghemanth's answer

View solution in original post

ghemanth
New Member

Thanks @BonMot.

It worked for me after re-arranging the 'eval' and 'top' paramters

 ERROR 
 | top limit=20 error_field,type 
 | eval type="day" 
 | append 
     [ search ERROR earliest=-7d 
         | top limit=20 error_field ,type 
         | eval type="week" 
     ] 
 | stats count,values(type) AS type by error_field  
 |  where count=1 and type="day"
0 Karma

BonMot
Explorer

That is actually cleaner. In that case you don't need to have type in your top command. I'll update my answer

0 Karma

BonMot
Explorer

Mutisearch isn't doing what you think. It is literally running both searches in parallel and returning the results interwoven as they are found. Since you have to wait for both searches to complete, you may as well append and count up the results Here is a version I did to compare security alerts today vs last 7 days. The first search time picker is "Today"

ERROR 
| top limit=20 error_field
| eval type="day" 
| append 
    [ search ERROR earliest=-7d 
        | top limit=20 error_field 
        | eval type="week"
    ] 
| stats count,values(type) AS type by error_field  
|  where count=1 and type="day"

What is going on here is we're labeling each search with the eval command, then getting the top error_field and type for each search and concatenating everything into one result. Finally, we run some stats to eliminate any error_fields in both search sets and what's left is the error_field and type of the unique entries of type "day".

NOTE: Edited to incorporate improvements from @ghemanth's answer

Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...