This appears to be a bug in splunk, as changing setting there seem to "trick" this into working.
If I set up a new slack alert to a #channel - it throws the 500's.
If I set up a slack alert with default (blank) channel, it triggers properly, and ~50% of the time, once a successful message was triggered to default endpoint, the alert can be set to a different #channel and it will trigger correctly.
I have not tried setting this up specifically with full admin permissions, however it seems a fairly large gap to have to grant full admin rights to something that simply need to post messages.
The Slack user owning the webhook may need elevated privileges in Slack (not Splunk.) Setting the account to 'admin' in Slack seems to work, but I would look into whether some kind of lesser permissions will also work.