Deployment Architecture

duplicate RT alerts in Search Head Cluster


We have an issue with quite a number of duplicate alerts, all but 1 are real time searches. One example there were 5 alerts at the exact same time from 5 of the 11 Search Head in the Search Head Cluster .

Any ideas?

Tags (1)
0 Karma

Splunk Employee
Splunk Employee

Here are few things to check:

1) Check if the delegation failed due to quota issues ? i.e with in the same minute we re-try twice..
2) Also, check if you see connectivity issue between search head cluster and Captain. It could be because the SH's running the alerts are not able to connect to captain
3)Check the health of the members during that 5 alert fired period, like is the captain switching too fast.
4) Check if the members are all part of the show shcluster-status and the RT jobs have duplicate among members.When a captain receives information about searches in the cluster and it finds duplicate RT searches they should be cancelled.

0 Karma
Get Updates on the Splunk Community!

Index This | Why do they call it hyper text?

November 2023 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

State of Splunk Careers 2023: Career Resilience and the Continued Value of Splunk

For the past three years, Splunk has partnered with Enterprise Strategy Group to conduct a survey that gauges ...

The Great Resilience Quest: 9th Leaderboard Update

The ninth leaderboard update (11.9-11.22) for The Great Resilience Quest is out >> Kudos to all the ...