topic Re: Why is my Splunk email trigger sending an email for every host found? in Alerting

Why is my Splunk email trigger sending an email for every host found?

mbrannaman — Wed, 13 Feb 2019 15:16:35 GMT

I created an alert with:

index=xxxx "Error Message"|
stats count as COUNT by host|
where COUNT > 6000

and an alert with the trigger condition where COUNT > 6000 and to send a message for each result.

The issue I have is that the alert returns only 2 records, but the alert sends an email for every single host.

How can I have emails sent for just the hosts returned?

Thanks!

Re: Why is my Splunk email trigger sending an email for every host found?

renjith_nair — Wed, 13 Feb 2019 15:45:50 GMT

@mbrannaman,

If you have the where condition already part of the search, remove it from the trigger condition and add No of Results is greater than 0

Re: Why is my Splunk email trigger sending an email for every host found?

mbrannaman — Wed, 13 Feb 2019 18:31:31 GMT

Thanks! But it still returns an email for every host regardless of the count.
The issue I have is (I think) the docs are pretty sparse, but it says the alert triggers off the base search - which does not include the count or the where.
I am not sure how to combine a count and where into the base search of
index=xxxx "Error Message"|
Any ideas?

Re: Why is my Splunk email trigger sending an email for every host found?

woodcock — Thu, 14 Feb 2019 01:01:32 GMT

I am very confused by the way you have phrased your question but in addition to what @renjith.nair said, you might need to change the alert's digest setting.

Re: Why is my Splunk email trigger sending an email for every host found?

woodcock — Thu, 14 Feb 2019 01:02:26 GMT

If it doesn't work, why did you click Accept?

Re: Why is my Splunk email trigger sending an email for every host found?

renjith_nair — Thu, 14 Feb 2019 04:11:06 GMT

Term base search might be confusing but whatever you are doing in your search above should work with the alert. Please make sure that the time frame you select for your search and alert are same, i.e. verify the alert configuration and check the time window/range.

Also, check the recent alert - view recent -> Jobs -> Actions > Job -> Inspect Job and check the search log. This should give you information about the search ran for the alert

Re: Why is my Splunk email trigger sending an email for every host found?

mbrannaman — Thu, 14 Feb 2019 13:38:57 GMT

I think I found the issue - I right clicked on the alert and opened the search in a separate panel to edit it, as well as the alert settings in the original pane.
Editing and saving the search in the new panel did not save it in the actual. Discovered that when I reopened the alert and saw it had not been saved for whatever reason.
Thanks for the help.