Splunk Search

How to write a search to alert if there is a growing number of a particular type of event?

Gayathirik
Path Finder

How to detect if there is a growing number of a particular type of event? It could indicate “flapping” on the Exchange server. I was thinking that a trendline could be made and then if the trend is going up in events day by day, this could detect it. If the number of events is twice the day before or the last few days, then an alert (email) can be sent out. Let me know if that makes sense.

Please provide me with a search that deals with the above requirement.

index=msexchange sourcetype="MSExchange:2013:HttpProxy" host="ftlpex02cas01.citrite.net" RpcHttp AND "/rpc/rpcproxy.dll"| 
0 Karma

sundareshr
Legend

Try this

**** UPDATED****

index=msexchange sourcetype="MSExchange:2013:HttpProxy" host="ftlpex02cas01.citrite.net" RpcHttp AND "/rpc/rpcproxy.dll" earliest=-1d@d | timechart span=1h count | eval day=strftime(_time, "%d") | evenstats avg(count) as hr_avg stdev(count) as hr_stdev by day | table _time count hr_avg hr_stdev | eval alert=if(count>=hr_stdev*2, "Y", "N")
0 Karma

Gayathirik
Path Finder

This query is errored out:

Error in "eventstats" command.The argument "stddev(count)" is invalid.

0 Karma

sjohnson_splunk
Splunk Employee
Splunk Employee

it's stdev(count) not stddev.

Gayathirik
Path Finder

"stdev" is giving 0.0000 value for all the days. the above search query is not providing the expected result.

the count should be the last 7 days but not the current day
it should be considered a "spike" if the count for one day is 2x the average

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...