You need a certificate that specifies the alt name, which doesn't happen when in the Splunk guide for cert creation.
One option would be to follow Step 3 in this guide. If you have a single server that creates all your certificates, you would need to change openssl.cnf before each cert creation. FYI, changing these type of files will cause a manifest error until you either put the old file back or upgrade.
https://www.hurricanelabs.com/splunk-tutorials/splunk-certificates-master-guide
#Edit the openSSL file
vi /opt/splunk/openssl/openssl.cnf
# Uncomment out the Request Extensions options
# Optional: Use "/" to search for req_extensions
Change FROM: #req_extensions = v3_req # The extensions to add to a certificate request
TO: req_extensions = v3_req # The extensions to add to a certificate request
# Optional: Use "/" to search for v3_req
#Add extended key usage 'subjectAltName = DNS:<FQDN>, DNS:<hostname>, IP:<ip_address>'
... View more