Okay, here is how you can do this. But be warned; you can easily break stuff here and also keep in mind that the default $SPLUNK_HOME/etc/apps/search/bin/sendemail.py can be over-written with any Splunk update.
So what must be done, to filter out alerts fired because a search failed on some search peer?
Backup the Splunk provided $SPLUNK_HOME/etc/apps/search/bin/sendemail.py and open in your favorite editor.
Go to line number 249 which contains the following:
body.write(intro)
this is the command to write the email content. Now we add the following lines JUST BEFORE line 249. What out for Python indentation , again: you can break stuff here!
# filter failed searches because of ..... foo bar
myList = ['Search generated the following messages', 'Search results may be incomplete']
for myError in myList:
if myError in intro:
admin = "
[email protected]"
recipients = []
message.replace_header('To', admin)
recipients.extend(EMAIL_DELIM.split(admin))
subject="Search failed, get up and check why...."
message['Subject'] = subject
Above lines will check intro for any value set in myList . If there is a match the email reciepient To will be changed to be the admin email set in admin . Also it changes the alert subject to what ever is set in subject above.
That's it, if you done everything correct only the defined admin will recieve those alerts.
Again, you've been warned and don't blame me if you break something.....
hope this helps ...
cheers, MuS
... View more
