How to troubleshoot why we are getting "appserver port: already bound" on startup of Splunk 6.2.x after reboot?

[arber]

Hello,

we are having the following issue while trying to start splunk (version 6.2) .

/opt/splunk/bin/splunk start

Splunk> Now with more code!

Checking prerequisites...
        Checking http port [8000]: open
        Checking mgmt port [8089]: open
        Checking appserver port [127.0.0.1:8065]: already bound
ERROR: The appserver port [127.0.0.1:8065] is already bound.  Splunk needs to use this port.
Would you like to change ports? [y/n]: y
Enter a new appserver port: 8066
Setting appserver port: 8066
The port specified is in use.  Please enter a different port.
Enter a new appserver port: 8067
Port "8067" is invalid, please enter a number between 1 and 65535
Enter a new appserver port:

The appserver port already bound.. We don't have anything on this system except Splunk. I also tried to check what process was using this port, but i didn't get any response. Also tried to set another port 8066 or 8067 but not possible. We also tried version 6.2.1 same issue.

The issue started after we did a reboot of the system

Any clue on this.

[splunkreal]

Hello,

same issue in 8.2.2 after upgrading some apps like Splunk_TA_nix or Splunk_TA_oracle, had to do this :

ps -ef | grep -i python

siem     11808     1  0 Apr18 ?        00:03:52 /OPT/siem/splunk/bin/python3.7 -O /OPT/siem/splunk/lib/python3.7/site-packages/splunk/appserver/mrsparkle/root.py --proxied=127.0.0.1,8065,8000

siem     34758 30871  0 12:26 pts/0    00:00:00 grep --color=auto -i python

[siem@s201lbasplsh2 ~]$ kill 11808

[mdessus_splunk]

I had the same issue: it was caused by the loopback not initialized. So check your interface with a ifconfig lo

[srubik]

I am also seeing this same issue and the only way I was able to get around it was by setting the port to 0 in web.conf. netstat showed port 8086 not in use, and I couldn't use any other port. Were you able to figure out why you were seeing this?

[chanfoli]

Can we see what the output of the following commands is after stopping splunk or making sure that splunk processes are not running with ps etcetera:

netstat -an | grep 8065

and

lsof |grep 8065

[arber]

Hi,

the Splunk is stopped. And both those commands do not show any result
It is very strange why it would say that the port is bound to anything

[hvaillancourt]

Did anyone ever figure this out?

[jsanderson]

In our case it turned out there was a BIND_IP defined in /opt/splunk/etc/splunk-launch.conf that was causing the issue (thought we had removed all of these).