Slack Notification Alert: Why are we getting "HTTP Error 500" in Splunk when sending an alert to a #channel instead of a @user?

[GeorgeStarkey]

Whenever we set up a Slack alert to send to a #channel in our Slack instance (other than default for the web hook), we get a 500 in Splunk.

07-08-2016 18:51:17.286 +0000 ERROR sendmodalert - action=slack STDERR -  Error sending message: HTTP Error 500: Server Error

Sending the alert destination to @user works fine, and leaving blank works fine for hitting the default channel

I can not find what is causing this behavior

[GeorgeStarkey]

This appears to be a bug in splunk, as changing setting there seem to "trick" this into working.

If I set up a new slack alert to a #channel - it throws the 500's.
If I set up a slack alert with default (blank) channel, it triggers properly, and ~50% of the time, once a successful message was triggered to default endpoint, the alert can be set to a different #channel and it will trigger correctly.

I have not tried setting this up specifically with full admin permissions, however it seems a fairly large gap to have to grant full admin rights to something that simply need to post messages.

[jspears]

The Slack user owning the webhook may need elevated privileges in Slack (not Splunk.) Setting the account to 'admin' in Slack seems to work, but I would look into whether some kind of lesser permissions will also work.