Splunk Search

How to set an alert that will trigger an email when based on these conditions?

AdixitSplunk
Path Finder

HI All,
I need some help in setting alerts for a condition, where I'm using a simple Splunk search to get whether the service is started for 3 hosts: Index=xyz sourcetype=123 Message="Started"|stats count by host Message|addcoltotals

Host  Message  Count
123   started  1
345   started  1
567   started  1
Total          3 

Now i want to set an alert condition which will trigger an email when :
a) AS soon as the count total is 3
b) If the count total is less than 3 from 5:10 -5:30 PM

If can be done in two ways one from alert setting and other editing in the search.

0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi AdixitSplunk
you could create a a search like this:

index=xyz sourcetype=123 Message="Started" (host=host1 OR host=host2 OR host=host3) 
| dedup host 
| stats count 
| where count<3

and so generate an alert.
if the servers number will grow, you can insert them in a lookup table:

index=xyz sourcetype=123 Message="Started" [ | impulookup server_lookup.csv | fields host ] 
| appendpipe [ | impulookup server_lookup.csv | stats count AS number]
| dedup host 
| stats values(number) AS number count 
| where count<number

You have only to decide the time period to use in the search and for the alert frequency.

Bye.
Giuseppe

View solution in original post

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi AdixitSplunk
you could create a a search like this:

index=xyz sourcetype=123 Message="Started" (host=host1 OR host=host2 OR host=host3) 
| dedup host 
| stats count 
| where count<3

and so generate an alert.
if the servers number will grow, you can insert them in a lookup table:

index=xyz sourcetype=123 Message="Started" [ | impulookup server_lookup.csv | fields host ] 
| appendpipe [ | impulookup server_lookup.csv | stats count AS number]
| dedup host 
| stats values(number) AS number count 
| where count<number

You have only to decide the time period to use in the search and for the alert frequency.

Bye.
Giuseppe

0 Karma

AdixitSplunk
Path Finder

HI Cusello ,
I have this query can we set up a alert and send 2 separate mails as per the condition in the query .
index=xyz sourcetype=123 Message="Started" (host=host1 OR host=host2 OR host=host3)
| dedup host
| stats count
| where count <=3
One mail if count is < 3 and one mail when count is = 3 .
Is this possible 😐

0 Karma

somesoni2
Revered Legend

What is the time range that you use for your search? How frequently those service start/stop?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...