Splunk Search

How to write a search to alert if there is a growing number of a particular type of event?

Path Finder

How to detect if there is a growing number of a particular type of event? It could indicate “flapping” on the Exchange server. I was thinking that a trendline could be made and then if the trend is going up in events day by day, this could detect it. If the number of events is twice the day before or the last few days, then an alert (email) can be sent out. Let me know if that makes sense.

Please provide me with a search that deals with the above requirement.

index=msexchange sourcetype="MSExchange:2013:HttpProxy" host="ftlpex02cas01.citrite.net" RpcHttp AND "/rpc/rpcproxy.dll"| 
0 Karma

Legend

Try this

**** UPDATED****

index=msexchange sourcetype="MSExchange:2013:HttpProxy" host="ftlpex02cas01.citrite.net" RpcHttp AND "/rpc/rpcproxy.dll" earliest=-1d@d | timechart span=1h count | eval day=strftime(_time, "%d") | evenstats avg(count) as hr_avg stdev(count) as hr_stdev by day | table _time count hr_avg hr_stdev | eval alert=if(count>=hr_stdev*2, "Y", "N")
0 Karma

Path Finder

This query is errored out:

Error in "eventstats" command.The argument "stddev(count)" is invalid.

0 Karma

Splunk Employee
Splunk Employee

it's stdev(count) not stddev.

Path Finder

"stdev" is giving 0.0000 value for all the days. the above search query is not providing the expected result.

the count should be the last 7 days but not the current day
it should be considered a "spike" if the count for one day is 2x the average

0 Karma