Finally resolved the issue. The Splunk installation was at a client's site and was accessed over a VPN. The client also had a proxy inbetween me and the splunk server which messed up the SSL sessions and presented me with the weird certificate.
So, my problem had nothing to with Splunk misbehaving.
... View more