Trigger Alert on 10 consecutive occurrences of ErrorCode=

[irishmanjb]

Hello Splunkers

I have an IIS log I need to open and search through every 15 minutes. If I see 10 consecutive occurences of the field ErorrCode= within a five minute period I want to trigger an alert.

ErrorCode=  Will be populated with different error codes like this ErrorCode=T1234.  I dont want to count individual error codes just the field ErrorCode.  I have been able to seach for occurences using alerts but the requirement if that they have to be consecutive occurences.

Here is an example snippet  ErrorCode can return several different values.  We dont care about the individual codes we just want to trigger when  the term ErrorCode appears in 10 consecutive lines of the log during a time window of 5 minutes

LogTypeID="x", InfoSourceID="x", ErrorText="xxxxxxxx", ErrorCode="XXXXXX", ErrorDescription="xxxxxxxx", IISServerName="xxxxxx", CreatedDate="2020-08-14 10:19:34.557", CreatedBy="xxxxxx", MemberShipID="xxxxxx", RegisterRequestTime="2020-08-14 10:16:06.0", AppCode="xxxxxx", InvalidAppCode="xxxxxx,"

any help would be appreciated

thanks

John

[richgalloway · Accepted solution]

Try this.  It will trigger an alert when there are 10 or more consecutive errors, but will not return the individual events.

index = foo
| bin span=5m _time
| streamstats reset_after=(isnull(ErrorCode)) count
| where (count >= 10 AND isnotnull(ErrorCode))

[irishmanjb]

sorry for the delay this set me down the right path thanks!

[to4kawa]

by clause is missing.

[irishmanjb]

thanks to4kawa on what answer?

[irishmanjb]

thanks will test this.

[venkateshparank]

@irishmanjb 
index=abc ErrorCode=* |bin span=5m _time | stats count by ErrorCode,_time|where count>=10

Try above query. It should work.

[to4kawa]

If the event doesn't have ErrorCode, this query can't work.

[to4kawa]

streamstats has time_window option.
but there is not the detail of consecutive occurrences .

[irishmanjb]

thanks will take a look