Splunk Search

finding greater than value from search resutls

gerald_huddlest
Path Finder

hi
I have created an eventtype that looks for a certain event across 12 servers (cmchost). I created a dashboard showing showing the number of events per cmchost over time.

I would like to set a threshold to alert when the number of events per server in a 15 minute period is exceeded but am struggling to put this last part together.

Dashboard search is:
tag=failure | dedup _raw | timechart count by CmcHost

I want an alert when a cmchost exceeds 10 events in 15 minute period.

grateful for some advise

Tags (1)
0 Karma
1 Solution

eelisio2
Path Finder

This will return only the servers with more than 10 events:

tag=failure | dedup _raw | stats count by CmcHost | search count > 10

This will only return rows where the count is greater than 10.
Then, you can alert if number of events(rows returned by the search) is greater than zero.

View solution in original post

gerald_huddlest
Path Finder

thanks, exactly what i was looking for!

0 Karma

eelisio2
Path Finder

This will return only the servers with more than 10 events:

tag=failure | dedup _raw | stats count by CmcHost | search count > 10

This will only return rows where the count is greater than 10.
Then, you can alert if number of events(rows returned by the search) is greater than zero.

gerald_huddlest
Path Finder

thanks for the resposne, but that will look for the events generated by all 12 servers. I only want to alert if 1 server exceeds 10 events in a 15 minute period which i why i was putting hte results of the original search into a table so that I can see how many events per server.
I dont want to run a seperate search for each server which would be the easy way to do it, but want to combine into a single search.

0 Karma

Drainy
Champion

have a look at;
http://docs.splunk.com/Documentation/Splunk/4.3/User/SchedulingSavedSearches

Basically you don't need to do the count for it, just do the search to return the events. Through the scheduled search/alert screen set it to alert when the numnber of results exceeds 10 and schedule it to run over a 15 minute period or perhaps every 5 minutes for some overlap.

Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...