Deployment Architecture

duplicate RT alerts in Search Head Cluster


We have an issue with quite a number of duplicate alerts, all but 1 are real time searches. One example there were 5 alerts at the exact same time from 5 of the 11 Search Head in the Search Head Cluster .

Any ideas?

Tags (1)
0 Karma

Splunk Employee
Splunk Employee

Here are few things to check:

1) Check if the delegation failed due to quota issues ? i.e with in the same minute we re-try twice..
2) Also, check if you see connectivity issue between search head cluster and Captain. It could be because the SH's running the alerts are not able to connect to captain
3)Check the health of the members during that 5 alert fired period, like is the captain switching too fast.
4) Check if the members are all part of the show shcluster-status and the RT jobs have duplicate among members.When a captain receives information about searches in the cluster and it finds duplicate RT searches they should be cancelled.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In the last month, the Splunk Threat Research Team (STRT) has had 2 releases of new security content via the ...

Announcing the 1st Round Champion’s Tribute Winners of the Great Resilience Quest

We are happy to announce the 20 lucky questers who are selected to be the first round of Champion's Tribute ...

We’ve Got Education Validation!

Are you feeling it? All the career-boosting benefits of up-skilling with Splunk? It’s not just a feeling, it's ...