Alerting

Alert when there is a X% increase in all events during a given time period?

alvarezmj
Explorer

I want to create an alert that will notify me when there is a X% increase in the total amount of events in a time period (ie X% frequency increase). I can't seem to find out how to connect this data to pull this all together, specifically I cannot figure out how to connect the frequency of events during current time period against the total average # events per day. Any help is greatly appreciated.

masonmorales
Influencer

Awesome. SO I can just change the
"earliest=" condition to be 30d to
check the count against the daily
average over the last month?

Eh, you have to do a tiny bit more for that. The search you gave would be comparing the count for a full 30 days to today's count. If you want an average of the daily count for the last 30 days you need to do something like:

index=indexA sourcetype=sourcetypeA earliest=-30d@d latest=@d | bucket _time span=1d | stats count as count by _time | stats avg(count) as DailyAvgOfMonth | appendcols [search index=indexA sourcetype=sourcetypeA earliest=@d latest=now | stats count as Today ] | where Today>=1.5*DailyAvgOfMonth

masonmorales
Influencer

Also, I'm assuming you are doing the last 30 days to smooth out any spikes that may have occurred. In which case, you may even want to use a median instead of an average.

i.e.

index=indexA sourcetype=sourcetypeA earliest=-30d@d latest=@d | bucket _time span=1d | stats count as count by _time | stats median(count) as DailyAvgOfMonth | appendcols [search index=indexA sourcetype=sourcetypeA earliest=@d latest=now | stats count as Today ] | where Today>=1.5*DailyAvgOfMonth
0 Karma

somesoni2
Revered Legend

Here is one sample search for alert. For example, I am checking if the data logged for an index/sourcetype today has increated over 50% from the data indexed for same index/sourcetype yesterday.

index=indexA sourcetype=sourcetypeA earliest=-1d@d latest=@d | stats count as Yesterday | appendcols [search index=indexA sourcetype=sourcetypeA earliest=@d latest=now | stats count as Today ] | where Today>=1.5*Yesterday

Setup an alert if number of events from above search > 0

Updated

index=indexA sourcetype=sourcetypeA earliest=-30d@d latest=@d | timechart span=1d count | stats avg(count) as DailyAvgOfMonth | appendcols [search index=indexA sourcetype=sourcetypeA earliest=@d latest=now | stats count as Today ] | where Today>=1.5*DailyAvgOfMonth

Stephan
Engager

Hello,
how can I do this for multiple hosts seperatly?
In my case I want to look for increasing Windows Events for each server.
regards
Stephan

0 Karma

Stephan
Engager

thanks. got it.

used innerjoin instead of "appendcols" and I have to set "timechart ... limit=0"

0 Karma

StringBee
Explorer

Can you please mention query here, it will be helpful...

Tags (1)
0 Karma

alvarezmj
Explorer

Awesome. SO I can just change the "earliest=" condition to be 30d to check the count against the daily average over the last month?

index=indexA sourcetype=sourcetypeA earliest=-30d@d latest=@d | stats count as Month | appendcols [search index=indexA sourcetype=sourcetypeA earliest=@d latest=now | stats count as Today ] | where Today>=1.5*Month

0 Karma

alvarezmj
Explorer

The whole point is so that we can be notified that A. There was a massive increase in # logged events. B. React to a sudden increase accordingly or at least look into it

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...