Post Process search returning different results

[mstephenson716]

I'm attempting to optimize one of our dashboard forms with a scheduled report as a global search that would need to be tokenized and will end up feeding several panels. I was attempting to build the base search and move my filtering tokens further down the query but I'm getting different results than I'm expecting based on the original query. I've double checked the time picker, but I'm not sure what's going on. I'm pretty new to dashboard post processes and global searches, is there a best practice, or SPL notion that I'm missing?

I took the original search

| tstats summariesonly=true allow_old_summaries=true dc(Vulnerabilities.signature) as signature_count from datamodel=Vulnerabilities.Vulnerabilities where (Vulnerabilities.severity="critical" OR Vulnerabilities.severity="high") by Vulnerabilities.dest, _time
| rename "Vulnerabilities.dest" as dest, "Vulnerabilities.*" as "*"
| timechart span=7d avg(signature_count) as current_avg

I removed the timechart command so that I could see the underlying values of the original search. It returns 205,565 statics

| tstats summariesonly=true allow_old_summaries=true dc(Vulnerabilities.signature) as signature_count from datamodel=Vulnerabilities.Vulnerabilities where (Vulnerabilities.severity="critical" OR Vulnerabilities.severity="high") by Vulnerabilities.dest, _time
| rename "Vulnerabilities.dest" as dest, "Vulnerabilities.*" as "*"

My search with the filters moved down that generate 134,197 statistics

| tstats summariesonly=true allow_old_summaries=true dc(Vulnerabilities.signature) as signature_count from datamodel=Vulnerabilities.Vulnerabilities by Vulnerabilities.dest, _time, Vulnerabilities.severity
| rename "Vulnerabilities.dest" as dest, "Vulnerabilities.severity" as severity, "Vulnerabilities.*" as "*"
| search (severity="critical" OR severity="high")
| stats sum(signature_count) as signature_count by dest _time

[gcusello · Accepted solution]

Hi mstephenson716,
let me understand:

• you have a base search that's the second one of your question,
• in one panel you have ; | timechart span=7d avg(signature_count) as current_avg ,
• if you move the search condition ( | search (severity="critical" OR severity="high") ) from the base search to a panel, you have different number of results,

Is it correct?

At first one question: at the end of the base search do you reported all the fields to use in panels with the fields command?
in other words:

| tstats summariesonly=true allow_old_summaries=true dc(Vulnerabilities.signature) as signature_count from datamodel=Vulnerabilities.Vulnerabilities where (Vulnerabilities.severity="critical" OR Vulnerabilities.severity="high") by Vulnerabilities.dest, _time 
| rename "Vulnerabilities.dest" as dest, "Vulnerabilities.*" as "*"
| fields severity signature_count dest _time

Then check in the results of the base search (without filter) the values of severity, maybe after grouping the results are less.
Try using less resuts (a restricted time period) to debug the situation.

Ciao.
Giuseppe

[mstephenson716]

I followed your advice to start with a smaller data set and they appear to yield the same statistics but something changes once I get out to 5 days or more and my | search command is yielding fewer statistics. I wonder if I'm running into some kind of a limit time wise.

[gcusello]

try a restricted time period but fixed, not using latest=now, e.g. yesterday or last hour to be sure that events are always the same.
Ciao.
Giuseppe

[mstephenson716]

Giuseppe,

I'd done that and for small time periods my queries match up. I've put it to bed for now, but the time restriction, at least helped me determine if my searches were the same.

Thanks!