Splunk Search

How to write a search to alert if there is a growing number of a particular type of event?

Gayathirik
Path Finder

How to detect if there is a growing number of a particular type of event? It could indicate “flapping” on the Exchange server. I was thinking that a trendline could be made and then if the trend is going up in events day by day, this could detect it. If the number of events is twice the day before or the last few days, then an alert (email) can be sent out. Let me know if that makes sense.

Please provide me with a search that deals with the above requirement.

index=msexchange sourcetype="MSExchange:2013:HttpProxy" host="ftlpex02cas01.citrite.net" RpcHttp AND "/rpc/rpcproxy.dll"| 
0 Karma

sundareshr
Legend

Try this

**** UPDATED****

index=msexchange sourcetype="MSExchange:2013:HttpProxy" host="ftlpex02cas01.citrite.net" RpcHttp AND "/rpc/rpcproxy.dll" earliest=-1d@d | timechart span=1h count | eval day=strftime(_time, "%d") | evenstats avg(count) as hr_avg stdev(count) as hr_stdev by day | table _time count hr_avg hr_stdev | eval alert=if(count>=hr_stdev*2, "Y", "N")
0 Karma

Gayathirik
Path Finder

This query is errored out:

Error in "eventstats" command.The argument "stddev(count)" is invalid.

0 Karma

sjohnson_splunk
Splunk Employee
Splunk Employee

it's stdev(count) not stddev.

Gayathirik
Path Finder

"stdev" is giving 0.0000 value for all the days. the above search query is not providing the expected result.

the count should be the last 7 days but not the current day
it should be considered a "spike" if the count for one day is 2x the average

0 Karma
Get Updates on the Splunk Community!

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...

Updated Data Management and AWS GDI Inventory in Splunk Observability

We’re making some changes to Data Management and Infrastructure Inventory for AWS. The Data Management page, ...

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...