Splunk Search

How to write a search to alert if there is a growing number of a particular type of event?

Gayathirik
Path Finder

How to detect if there is a growing number of a particular type of event? It could indicate “flapping” on the Exchange server. I was thinking that a trendline could be made and then if the trend is going up in events day by day, this could detect it. If the number of events is twice the day before or the last few days, then an alert (email) can be sent out. Let me know if that makes sense.

Please provide me with a search that deals with the above requirement.

index=msexchange sourcetype="MSExchange:2013:HttpProxy" host="ftlpex02cas01.citrite.net" RpcHttp AND "/rpc/rpcproxy.dll"| 
0 Karma

sundareshr
Legend

Try this

**** UPDATED****

index=msexchange sourcetype="MSExchange:2013:HttpProxy" host="ftlpex02cas01.citrite.net" RpcHttp AND "/rpc/rpcproxy.dll" earliest=-1d@d | timechart span=1h count | eval day=strftime(_time, "%d") | evenstats avg(count) as hr_avg stdev(count) as hr_stdev by day | table _time count hr_avg hr_stdev | eval alert=if(count>=hr_stdev*2, "Y", "N")
0 Karma

Gayathirik
Path Finder

This query is errored out:

Error in "eventstats" command.The argument "stddev(count)" is invalid.

0 Karma

sjohnson_splunk
Splunk Employee
Splunk Employee

it's stdev(count) not stddev.

Gayathirik
Path Finder

"stdev" is giving 0.0000 value for all the days. the above search query is not providing the expected result.

the count should be the last 7 days but not the current day
it should be considered a "spike" if the count for one day is 2x the average

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...