topic Re: cliVerifyServerName failing because Splunk starts as 127.0.0.1 in Splunk Enterprise

Why cliVerifyServerName failing because Splunk starts as 127.0.0.1?

yaye — Mon, 14 Aug 2023 17:06:11 GMT

Hi,

I want to run the command "splunk reload deploy-server" on my deployment server, but it fails with the following error:

[root@server etc]# su splunk [splunk@server etc]$ splunk reload deploy-server Your session is invalid. Please login. ERROR: IP address 127.0.0.1 not in server certificate. Please see server.conf/[sslConfig]/cliVerifyServerName for details. Couldn't request server info: Couldn't complete HTTP request: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed

I'm running Splunk Enterprise 9.0.4.

The deployment server also acts as a license server and monitoring console.

Of course, my certificate does not have the localhost IP in it.

My Splunk has a Systemd Unit File.

#This unit file replaces the traditional start-up script for systemd #configurations, and is used when enabling boot-start for Splunk on #systemd-based Linux distributions. [Unit] Description=Systemd service file for Splunk, generated by 'splunk enable boot-start' After=network-online.target Wants=network-online.target [Service] Type=simple Restart=always ExecStart=/data/splunk/bin/splunk _internal_launch_under_systemd KillMode=mixed KillSignal=SIGINT TimeoutStopSec=360 LimitNOFILE=65536 LimitRTPRIO=99 SuccessExitStatus=51 52 RestartPreventExitStatus=51 RestartForceExitStatus=52 User=splunk Group=splunk Delegate=true CPUShares=1024 MemoryLimit=24949776384 PermissionsStartOnly=true ExecStartPost=-/bin/bash -c "chown -R splunk:splunk /sys/fs/cgroup/cpu/system.slice/%n" ExecStartPost=-/bin/bash -c "chown -R splunk:splunk /sys/fs/cgroup/memory/system.slice/%n" [Install] WantedBy=multi-user.target

sslConfig Part of my server.conf

[sslConfig] useClientSSLCompression = true sslVersions = tls1.2 sslVerifyServerCert = true sslVerifyServerName = true requireClientCert = false serverCert = <Combined PEM Cert> sslRootCAPath = <Root CA PEM Cert> sslPassword = <Password> cliVerifyServerName = true

If you need any more info, let me know.

Re: cliVerifyServerName failing because Splunk starts as 127.0.0.1

isoutamo — Fri, 11 Aug 2023 09:43:31 GMT

is this old environments (pre 9) which has upgraded?

Probably you have set mgmtHostPort to 127.0.0.1 on web.conf ?

You should check it by

splunk btool web list settings --debug|egrep mgmtHostPort $SPLUNK_HOME/etc/system/default/web.conf mgmtHostPort = 127.0.0.1:8089

If this is case, then try to remove is or bind this to real ip for FQDN which you have on your certificate.

mgmtHostPort = <string> * The host port of the splunkd process. * The IP address and host port where Splunk Web looks for the splunkd process. * The port listens on all avalible host IP addresses (0.0.0.0) * Don't include "http[s]://" when specifying this setting. Only include the IP address and port. * Default (on universal forwarders): localhost:8089 * Default (on all other Splunk platform instance types): 0.0.0.0:8089

r. Ismo

Re: cliVerifyServerName failing because Splunk starts as 127.0.0.1

yaye — Mon, 14 Aug 2023 09:58:57 GMT

We use DHCP for our servers, so it could be possible that after the lease time ends, the server has a new IP.

Is it possible to give mgmtHostPort a FQDN / CName?

Re: cliVerifyServerName failing because Splunk starts as 127.0.0.1

yaye — Mon, 14 Aug 2023 18:29:17 GMT

Yes, this is a upgraded environment.

Re: cliVerifyServerName failing because Splunk starts as 127.0.0.1

isoutamo — Tue, 15 Aug 2023 07:55:16 GMT

If that FQDN is always resolvable, then it should work. It's just like localhost vs. 127.0.0.1 which (localhost) is currently default.

If this is server then static IP would be better (IMHO). Of course if it's e.g. in AWS then you just need to update r53 automatically to keep ip vs FQDN in sync. Other option is add secondary interface with static ip.

Re: cliVerifyServerName failing because Splunk starts as 127.0.0.1

yaye — Wed, 16 Aug 2023 09:22:10 GMT

Changed the mgmtHostPort to the FQDN of the server.

This FQDN stands in the Certificate Subject Alternative Name of the certificate of the server.

[settings]

startwebserver = 1

mgmtHostPort = <redacted>:8089

enableSplunkWebSSL = true

privKeyPath = <redacted>

serverCert = <redacted>

sslVersions = tls1.2

max_upload_size = 2048

Changed cliVerifyServerName to true

[sslConfig]

useClientSSLCompression = true

sslVersions = tls1.2

sslVerifyServerCert = true

sslVerifyServerName = true

requireClientCert = false

serverCert = <redacted>

sslRootCAPath = <redacted>

sslPassword = <redacted>

cliVerifyServerName = true

Getting the same error after restarting the server

[root@<redacted> ~]# systemctl restart Splunkd

[root@<redacted> ~]# su splunk

[splunk@<redacted> root]$ splunk reload deploy-server

Your session is invalid. Please login.

ERROR: IP address 127.0.0.1 not in server certificate. Please see server.conf/[sslConfig]/cliVerifyServerName for details.

Couldn't request server info: Couldn't complete HTTP request: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed

Re: cliVerifyServerName failing because Splunk starts as 127.0.0.1

FlorianScho — Mon, 23 Oct 2023 13:18:27 GMT

You got that fixed?

Re: cliVerifyServerName failing because Splunk starts as 127.0.0.1

yaye — Mon, 20 Nov 2023 10:29:19 GMT

No, unfortunately I didn't get it solved.

But I didn't spend any more time on the problem.

Re: Why cliVerifyServerName failing because Splunk starts as 127.0.0.1?

orezaie — Mon, 13 Oct 2025 12:40:34 GMT

Here is the best answer:
To resolve the issue, please log in to Splunk and reload the deploy server:

$ /opt/splunk/bin/splunk login

Splunk username: admin # (use your actual username)

Password:

$ /opt/splunk/bin/splunk reload deploy-server -class XYZ

This will reload the specified server class(es).