Has anyone built an enhanced email alerting app?

[Lowell]

The built-in email alerting functionality is fairly limited to the degree at which you can customize or control the content of the email itself. There's lots of stuff I'd like to be able to do that I can't right now, for example:

• Issue a "CC" (or BCC) instead of just "TO". (I'd also like the ability to note splunk user names, and then let splunk convert that to their listed email address.)
• Automatically show "_time" as something humanly readable. (I know there are timezone issues here.)
• Customize or control the layout or content of the email body.
• I'd like to be able to hide (or push to the bottom the actual search string.) Because (1) non-technical or non-splunk users just find that confusing, and (2) for me, I have some long complicated searches with just a small table; so I have to scroll around for awhile on my blackberry just to get to the results table. *. I'd like to be able to add some simple template text to the output. (Or perhaps, just include the searches "description". We have several alerts that need additional explication.)
• I'd like to be able to alter the subject line independently of the search name. It would also be nice to include some value from the results within the subject. (Something like "Disk Error on host $results[0].host$").
• Modify the from address per search.
• Add links to splunk views. Either within the body of the email, or within the table results (for a per-row hyperlink).
• Including more than one search would be helpful from time to time. (Sure you can email PDF views, but they are pretty limited, and they load much slower than HTML content; especially on mobile devices.)

That's just off the top of my head. I know I can go hack the sendemail.py but that's a long-term maintenance nightmare.

Has anybody build some kind of app that augments/hijacks/replaces the out-of-the-box email alerting functionality?

[cpetterborg]

The Splunk app Red Alert could be used as a preprocessor of the data, and then you can easily set up a script that can send out the email in any format that you would like, including HTML email. We aren't using Red Alert this way ourselves, but we are using it to pass the data from the alert to our Event Management System (EMS) for use by the NOC. Red Alert was fairly simple to set up and the programming for the email could be done in any language you desire (except pig-latin).

[e2eadmin]

Would encrypting the email alerts with a digital certificate require writing a custom script or modifying sendmail.py? Any easy way to do this before the mail leaves the Indexer? Thanks.

[bmunson_splunk]

Splunk have listened.

Version 6.1 of splunk now has TO: CC: & BCC:, Priority, Subject and a multi line Message.
You also have the option of including the search string or not as well as the results.
And they have listed the tokens (like $alert.severity$) that can be used.

And this can be triggered from the search string with the sendmail command.

See http://docs.splunk.com/Documentation/Splunk/6.1.2/Alert/Setupalertactions#Email_notification for details.

[slierninja]

Email Alerts are much improved - thanks for baking this in! The tokens are a nice touch also.

One thing I think could be added to this is HTML messaging support in the body (message) or footer. It seems to be escaping any HTML characters I add for providing a richer email messing interface.

Example: message="This is an example <b>message</b> Splunk"

Any idea why the default email server under Settings-->Server Settings-->Email Settings-->Mail Host isn't used? It seems to be require that you provide it with each sendemail request which seems odd since there is a default config.

Thanks!

[Lowell]

BTW, I did end up making a custom version of the sendemail.py script, and it did make upgrade more complicated. The features I focused on were: 1.) Allowing custom intro text (which came from the "Description" field.) and 2.) I re-arranged the body of the email a bit to move the search link to the bottom (so that message previews would show some of the details. Maybe someday I'll write a full blown app. Of course first I need this to work: http://answers.splunk.com/answers/3110/can-you-add-your-own-custom-alert-action

[adamw]

We've done something similar by writing a scripted alert action script that takes in the results of a search, parse it, then send the email using the script itself.

See http://docs.splunk.com/Documentation/Splunk/latest/admin/Configurescriptedalerts#Script_options for details.

Beyond that, it might be useful to put in a feature request.

[brettcave]

+1

Definitely something that would add value, and I don't think it would be too hard to implement:

1. add a check box and display text area when checked. This option could override other options if necessary.
2. In sendemail.py, if the option was checked, set intro the the value of the text area (with variable merging).

Over and above changes to emails, it would make it very easy to configure SMS alerts in splunk, a real value add.