Alerting

Trigger Alert on 10 consecutive occurrences of ErrorCode=

irishmanjb
Path Finder

Hello Splunkers

I have an IIS log I need to open and search through every 15 minutes. If I see 10 consecutive occurences of the field ErorrCode= within a five minute period I want to trigger an alert.

ErrorCode=  Will be populated with different error codes like this ErrorCode=T1234.  I dont want to count individual error codes just the field ErrorCode.  I have been able to seach for occurences using alerts but the requirement if that they have to be consecutive occurences.

 

Here is an example snippet  ErrorCode can return several different values.  We dont care about the individual codes we just want to trigger when  the term ErrorCode appears in 10 consecutive lines of the log during a time window of 5 minutes

LogTypeID="x", InfoSourceID="x", ErrorText="xxxxxxxx", ErrorCode="XXXXXX", ErrorDescription="xxxxxxxx", IISServerName="xxxxxx", CreatedDate="2020-08-14 10:19:34.557", CreatedBy="xxxxxx", MemberShipID="xxxxxx", RegisterRequestTime="2020-08-14 10:16:06.0", AppCode="xxxxxx", InvalidAppCode="xxxxxx,"

 

any help would be appreciated

thanks

John

Labels (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Try this.  It will trigger an alert when there are 10 or more consecutive errors, but will not return the individual events.

index = foo
| bin span=5m _time
| streamstats reset_after=(isnull(ErrorCode)) count
| where (count >= 10 AND isnotnull(ErrorCode))
---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Try this.  It will trigger an alert when there are 10 or more consecutive errors, but will not return the individual events.

index = foo
| bin span=5m _time
| streamstats reset_after=(isnull(ErrorCode)) count
| where (count >= 10 AND isnotnull(ErrorCode))
---
If this reply helps you, Karma would be appreciated.

irishmanjb
Path Finder

sorry for the delay this set me down the right path thanks!

Tags (1)
0 Karma

to4kawa
Ultra Champion

by clause is missing.

0 Karma

irishmanjb
Path Finder

thanks to4kawa on what answer?

0 Karma

irishmanjb
Path Finder

thanks will test this.

0 Karma

venkateshparank
Path Finder

@irishmanjb 
index=abc ErrorCode=* |bin span=5m _time | stats count by ErrorCode,_time|where count>=10

Try above query. It should work.

0 Karma

to4kawa
Ultra Champion

If the event doesn't have ErrorCode, this query can't work.

to4kawa
Ultra Champion

streamstats has time_window option.
but there is not the detail of consecutive occurrences .

0 Karma

irishmanjb
Path Finder

thanks will take a look

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...