Changing SERVER HEALTH ALERT emails

[Hazel]

Hi,

We get many alerts sent to us about cpu health under the email heading SERVER HEALTH ALERT - followed by tags. These run the search such as

Query Terms: 'index="main" host="glon12u10001" sourcetype="WMI:CPUTime"' or Query Terms: 'index="os" host="sos45a-4104*" source="cpu"' etc

I can't find out where these alerts are configured. They are not in the list under Searched and Reports - I want to change who these are sent to. Does anyone know where these would be configured?

I have also noticed that although we have changed our tags and the changes have made it through to the web search, the email alerts still display the old tags - are these configured somewhere different? The old tags do not appear in the list of Tags setup, i have also done a find command on the indexer for tags.conf, and no files contain the tags it is using. Does anyone know where this is configured?

[gkanapathy · Accepted solution]

They should be under Searches and Reports. Possibilities are that they are in a different app (make sure you're viewing all apps in the UI), or there's another Splunk server (possibly a distributed node or search head) running the searches. That would also explain the strange tags.

[Hazel]

Thankyou for your answers, I have found the alerts under a different application - didn't realise that the Searches & Reports page defaults this to the search app only.

[jrodman]

Do alerts say who sent them? You should be able to figure this out with the email headers, but it seems like it should probably just be in there by default.

[Lowell]

I don't think these are standard saved searches. What all splunk apps have you installed?