I need some help in setting alerts for a condition, where I'm using a simple Splunk search to get whether the service is started for 3 hosts:
Index=xyz sourcetype=123 Message="Started"|stats count by host Message|addcoltotals
Host Message Count 123 started 1 345 started 1 567 started 1 Total 3
Now i want to set an alert condition which will trigger an email when :
a) AS soon as the count total is 3
b) If the count total is less than 3 from 5:10 -5:30 PM
If can be done in two ways one from alert setting and other editing in the search.
What is the time range that you use for your search? How frequently those service start/stop?
you could create a a search like this:
index=xyz sourcetype=123 Message="Started" (host=host1 OR host=host2 OR host=host3) | dedup host | stats count | where count<3
and so generate an alert.
if the servers number will grow, you can insert them in a lookup table:
index=xyz sourcetype=123 Message="Started" [ | impulookup server_lookup.csv | fields host ] | appendpipe [ | impulookup server_lookup.csv | stats count AS number] | dedup host | stats values(number) AS number count | where count<number
You have only to decide the time period to use in the search and for the alert frequency.
HI Cusello ,
I have this query can we set up a alert and send 2 separate mails as per the condition in the query .
index=xyz sourcetype=123 Message="Started" (host=host1 OR host=host2 OR host=host3)
| dedup host
| stats count
| where count <=3
One mail if count is < 3 and one mail when count is = 3 .
Is this possible 😐