This seems to be the only place where this information is to be found, thanks @cbtadmin!
It can be checked like this:
/splunk cmd openssl x509 -text -in /opt/splunk/etc/auth/your_server_cert_and_key.pem
You should see a line like this:
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication
... View more