SPlunk not sending alerts

[SS1]

Hi,

My splunk instance is not sending email alerts for a new alert th Can soat i just setup. I am getting other alert emails from the same splunk instance but the new alert isn't sending the alerts although it generates a stats table. I have set the alert to trigger per result. Same alert trigger condition works for other results. Any help is appreciated.

[SS1]

Hi,

@ITWhisperer 

We are running on 8.0.2

Below is the non working alert (working alert setup is same but the query doesnt have 'eval' )

Below is the query (it is generating stats table) ,

(sourcetype="AppV-User" *PUT /package*) OR (sourcetype=sql_appv_latest)
| rex "\/packages\/\w+\/(?<Id>\w+)\s+-\s+\d+\s+(?<User>[^ ]+)"
| eval Id=coalesce(Id, ID)
| stats dc(sourcetype) AS dc_sourcetype values(PkgName) as PkgName values(user) as user values(Status) as Status by Id
| where dc_sourcetype=2

ALert Setup,

[ITWhisperer]

Have you tried other trigger conditions such as number of results greater than 0?

[ITWhisperer]

Can you share the alerts, particularly, how the queries and triggers are set up for working and non-working alerts? Also, which version of splunk are you using? We have had issues with custom triggers not firing correctly so perhaps it is related to that, but seeing what you have working and not working would be helpful.

[burwell]

Hello. This sounds like the bug  we hit in 8.0.2

When we had an alert and then used custom criteria for alerting, if we had.. 

search count > 0

Splunk stored search count &gt; 0 in the savedsearches.conf and then the alert never fired.

This bug was fixed in 8.0.5: https://docs.splunk.com/Documentation/Splunk/8.0.5/ReleaseNotes/Fixedissues
SPL-189917

If you do an advanced edit, I believe you can see the bug. 

If this is your case, the solution would be to upgrade to 8.0.8.

I would not recommend 8.0.5 as it has this nasty bug 

2020-07-15

SPL-192057, SPL-188608

Realtime and in-progress adhoc searches shows "Job terminated unexpectedly" on members of SHC other than the SH from which the search originated

We upgraded to 8.0.2 then 8.0.5 and then 8.0.6.