https://community.splunk.com/t5/Deployment-Architecture/Why-am-I-unable-to-start-2-of-4-search-heads-with-quot-WARNING/m-p/216316#M8059 <P>Are you running all instances on same machine? If that's the case make sure your ports are not conflicting.</P> <P>Check splunkd.log for errors. One of your configuration might be incorrect or there are some conflicts.</P> <P>Also check if mongod instance (kvstore) is shutdown properly and no process is running. We had an issue where stale process of mongo was running which prevents re-starting.</P> <P>As always, log files are the first place to look for any issues.</P> Thu, 07 Jan 2016 03:26:14 GMT renjith_nair 2016-01-07T03:26:14Z https://community.splunk.com/t5/Deployment-Architecture/Why-am-I-unable-to-start-2-of-4-search-heads-with-quot-WARNING/m-p/216315#M8058 <P>I have four search head instances running in our NONPROD environment, but am getting the following error below when starting two of the four instances.</P> <P>We are running Splunk Enterprise Splunk 6.2.0 (build 237341)</P> <P>Regards,<BR /> -James O'Connell</P> <P>...</P> <PRE><CODE> All preliminary checks passed. Starting splunk server daemon (splunkd)... Done [ OK ] Waiting for web server at <A href="http://127.0.0.1:38000" target="test_blank">http://127.0.0.1:38000</A> to be available WARNING: web interface does not seem to be available! </CODE></PRE> Wed, 06 Jan 2016 22:52:26 GMT https://community.splunk.com/t5/Deployment-Architecture/Why-am-I-unable-to-start-2-of-4-search-heads-with-quot-WARNING/m-p/216315#M8058 jamesoconnell 2016-01-06T22:52:26Z https://community.splunk.com/t5/Deployment-Architecture/Why-am-I-unable-to-start-2-of-4-search-heads-with-quot-WARNING/m-p/216316#M8059 <P>Are you running all instances on same machine? If that's the case make sure your ports are not conflicting.</P> <P>Check splunkd.log for errors. One of your configuration might be incorrect or there are some conflicts.</P> <P>Also check if mongod instance (kvstore) is shutdown properly and no process is running. We had an issue where stale process of mongo was running which prevents re-starting.</P> <P>As always, log files are the first place to look for any issues.</P> Thu, 07 Jan 2016 03:26:14 GMT https://community.splunk.com/t5/Deployment-Architecture/Why-am-I-unable-to-start-2-of-4-search-heads-with-quot-WARNING/m-p/216316#M8059 renjith_nair 2016-01-07T03:26:14Z https://community.splunk.com/t5/Deployment-Architecture/Why-am-I-unable-to-start-2-of-4-search-heads-with-quot-WARNING/m-p/216317#M8060 <P>When I've had this issue, I chalked it up to entropy on the system over time. Assuming the config and the ports are all good, then I start with some simple and basic things:</P> <OL> <LI>Run <CODE>splunk stop</CODE> and then do a <CODE>kill -9</CODE> on the results of: <CODE>ps -ef | grep -i splunk | grep -v grep | awk '{print $2;}'</CODE> to stop anything that didn't actually shut down</LI> <LI>Restart the actual server thereby purging any zombie processes that might not be needed but are still running from usage over time</LI> <LI>When all else fails, reinstall that instance. This can be made easy through proper use of the configuration distribution features </LI> </OL> Thu, 07 Jan 2016 13:32:55 GMT https://community.splunk.com/t5/Deployment-Architecture/Why-am-I-unable-to-start-2-of-4-search-heads-with-quot-WARNING/m-p/216317#M8060 sloshburch 2016-01-07T13:32:55Z