Splunk Search

Exclude data if field stats percentage exceeded

arm3n
New Member

Hi folks, been banging my head against this for hours and am sure I am missing something obvious.  I have tried using eval and evenstats in various iterations, but no dice.

Essentially I am trying to have the below Alert/Search executed (which compares a volume of errors from one minute to the next).  The problem we've been running into, is that we have a field value SESSID that is assigned to unique sessions, and at times one "stuck" session can really muddy up this valuable canary alert for us.  My goal is to either exclude all results with an offending SESSID (e.g. where its stats percentage exceeded 10%) or just not trigger the alert for that particular minute.

 

 

host=app* LOGLEVEL=ERROR earliest=-2m@m latest=-1m@m | stats count as LastMinute | join host [search host=app* LOGLEVEL=ERROR earliest=-3m@m latest=-2m@m |bucket _time span=1m | stats count as PrevMinute ] | eval HigherThanPrevMinute=(3*PrevMinute) | where LastMinute > HigherThanPrevMinute |where LastMinute > 1000  |table LastMinute,PrevMinute

 

 

 

Thank you in advance!

-Armen

 

 

Labels (2)
0 Karma

skoelpin
SplunkTrust
SplunkTrust

Don't use join or do any subsearching.. Use `streamstats` instead!

Not exactly sure where you're going with the query, but this shows you how to find the previous value using `streamstats` in addition to find a percentage using `eventstats`

 

 

host=app* LOGLEVEL=ERROR 
| bin _time span=1m
| stats count as LastMinute by _time
| streamstats current=f last(LastMinute) AS prev_LastMinute
| eventstats sum(LastMinute) AS total
| eval percent=LastMinute/total

 

 

0 Karma

arm3n
New Member

Thank you for looking at my question, @skoelpin !  I do appreciate the improved simplicity of using streamstats and eventstats in lieu of my clunky join.  However, I am still interested in a way to identify a field value that has an unnaturally high percentage and remove its data from the results altogether.   In simplest terms:

If I have a thousand events coming back, and of those one unique SESSIONID value is encompassing 800 (80%) of the total SESSIONID values, I would only like to see the remaining 200 values for my result set.

Thanks again!

-Armen

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...