Splunk Search

Exclude data if field stats percentage exceeded

arm3n
New Member

Hi folks, been banging my head against this for hours and am sure I am missing something obvious.  I have tried using eval and evenstats in various iterations, but no dice.

Essentially I am trying to have the below Alert/Search executed (which compares a volume of errors from one minute to the next).  The problem we've been running into, is that we have a field value SESSID that is assigned to unique sessions, and at times one "stuck" session can really muddy up this valuable canary alert for us.  My goal is to either exclude all results with an offending SESSID (e.g. where its stats percentage exceeded 10%) or just not trigger the alert for that particular minute.

 

 

host=app* LOGLEVEL=ERROR earliest=-2m@m latest=-1m@m | stats count as LastMinute | join host [search host=app* LOGLEVEL=ERROR earliest=-3m@m latest=-2m@m |bucket _time span=1m | stats count as PrevMinute ] | eval HigherThanPrevMinute=(3*PrevMinute) | where LastMinute > HigherThanPrevMinute |where LastMinute > 1000  |table LastMinute,PrevMinute

 

 

 

Thank you in advance!

-Armen

 

 

Labels (2)
0 Karma

skoelpin
SplunkTrust
SplunkTrust

Don't use join or do any subsearching.. Use `streamstats` instead!

Not exactly sure where you're going with the query, but this shows you how to find the previous value using `streamstats` in addition to find a percentage using `eventstats`

 

 

host=app* LOGLEVEL=ERROR 
| bin _time span=1m
| stats count as LastMinute by _time
| streamstats current=f last(LastMinute) AS prev_LastMinute
| eventstats sum(LastMinute) AS total
| eval percent=LastMinute/total

 

 

0 Karma

arm3n
New Member

Thank you for looking at my question, @skoelpin !  I do appreciate the improved simplicity of using streamstats and eventstats in lieu of my clunky join.  However, I am still interested in a way to identify a field value that has an unnaturally high percentage and remove its data from the results altogether.   In simplest terms:

If I have a thousand events coming back, and of those one unique SESSIONID value is encompassing 800 (80%) of the total SESSIONID values, I would only like to see the remaining 200 values for my result set.

Thanks again!

-Armen

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...