Splunk services are running well but the webpage localhost:8000 can't be open

[jackywsy]

HI experts,

Below is the logs of Splunkd.log and web-service.log ,I clearly see the splunkd service is working well .And I have tried to restart the services. No errors occured during the actions. Can I know why this happened? I have tried to re-install it ,it is fine to go the page http://localhost:8000 and the port 8089 can't be connected . Is it caused by the license or not ? How can I re-open the webpage tomorrow if the problem was caused by the license? Just my guest ,500MB per day as we all known. I just try the trial verison for now .

My Operation System is Windows 2008 R2 , splunk Version 6.2

Thanks in advance, guys.

04-01-2015 16:54:08.424 +0800 WARN TimeoutHeap - Either time adjusted forwards by, or event loop was descheduled for 612234ms.
04-01-2015 17:13:08.842 +0800 INFO TcpInputProc - Stopping IPv4 port 514
04-01-2015 17:13:08.842 +0800 INFO TcpInputProc - Stopping IPv4 port 9997
04-01-2015 17:13:08.842 +0800 WARN TcpInputProc - Stopping all listening ports. Queues blocked for more than 300 seconds
04-01-2015 17:16:48.813 +0800 WARN HttpListener - Socket error from 127.0.0.1 while idling: error:1407609C:SSL routines:SSL23_GET_CLIENT_HELLO:http request
04-01-2015 17:16:48.813 +0800 WARN HttpListener - Socket error from 127.0.0.1 while idling: error:1407609C:SSL routines:SSL23_GET_CLIENT_HELLO:http request
04-01-2015 17:16:48.813 +0800 WARN HttpListener - Socket error from 127.0.0.1 while idling: error:1407609C:SSL routines:SSL23_GET_CLIENT_HELLO:http request
04-01-2015 17:16:53.890 +0800 WARN HttpListener - Socket error from 127.0.0.1 while idling: error:1407609C:SSL routines:SSL23_GET_CLIENT_HELLO:http request
04-01-2015 17:17:23.907 +0800 WARN HttpListener - Socket error from 127.0.0.1 while idling: error:1407609C:SSL routines:SSL23_GET_CLIENT_HELLO:http request
04-01-2015 17:18:03.022 +0800 WARN HttpListener - Socket error from 192.168.72.27 while idling: Read Timeout
04-01-2015 17:18:23.943 +0800 WARN HttpListener - Socket error from 127.0.0.1 while idling: error:1407609C:SSL routines:SSL23_GET_CLIENT_HELLO:http request
04-01-2015 17:22:09.050 +0800 WARN HttpListener - Socket error from 192.168.72.27 while idling: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
04-01-2015 17:23:24.063 +0800 WARN HttpListener - Socket error from 127.0.0.1 while idling: error:1407609C:SSL routines:SSL23_GET_CLIENT_HELLO:http request
04-01-2015 17:33:24.134 +0800 WARN HttpListener - Socket error from 127.0.0.1 while idling: error:1407609C:SSL routines:SSL23_GET_CLIENT_HELLO:http request

04-01-2015 16:43:56.970 +0800 INFO WatchedFile - Will begin reading at offset=23434 for file='C:\Program Files\Splunk\var\log\splunk\licensealert.log'.
04-01-2015 16:43:56.986 +0800 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='C:\Program Files\Splunk\var\log\splunk\license_audit.log'.
04-01-2015 16:43:57.189 +0800 INFO WatchedFile - Will begin reading at offset=4557061 for file='C:\Program Files\Splunk\var\log\splunk\metrics.log'.
04-01-2015 16:43:57.236 +0800 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='C:\Program Files\Splunk\var\log\splunk\paladin.log'.
04-01-2015 16:43:57.251 +0800 INFO WatchedFile - Will begin reading at offset=3979 for file='C:\Program Files\Splunk\var\log\splunk\python_modular_input.log'.
04-01-2015 16:43:57.267 +0800 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='C:\Program Files\Splunk\var\log\splunk\remote_searches.log'.
04-01-2015 16:43:57.267 +0800 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='C:\Program Files\Splunk\var\log\splunk\scHeadlinesHandler.log'.
04-01-2015 16:43:57.282 +0800 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='C:\Program Files\Splunk\var\log\splunk\searchhistory.log'.
04-01-2015 16:43:57.314 +0800 INFO WatchedFile - Will begin reading at offset=31477 for file='C:\Program Files\Splunk\var\log\splunk\splunkd-utility.log'.
04-01-2015 16:43:57.953 +0800 WARN HandleJobsDataProvider - Unable to save search history for user=nobody, app=splunk_app_microsoft_exchange, sid=1427877837.1, search='search index=_internal source=metrics.log group="per_sourcetype_thruput" (\n series="msexchange::topology" OR \n series="msexchange::messagetracking" OR \n series="msexchange::mailbox-usage" OR \n series="msexchange::folder-usage" OR \n series="msexchange::database-stats" OR \n series="msexchange::publicfolder-stats" OR \n series="msexchange::rpcclientaccess" OR \n series="msexchange::adminaudit" OR \n series="msexchange::throttlingpolicy" OR \n series="msexchange::mailboxaudit" OR \n series="msexchange::inboxrules" OR \n series="msexchange:*:distributionlists" OR \n series="msexchange:reputation"\n ) earliest=-1d@d latest=@d \n | stats sum(kb) as totKb by series \n | eval totB=totKb*1024 \n | stats sum(totB) as totB'
04-01-2015 16:43:59.466 +0800 WARN IniFile - C:\Program Files\Splunk\etc\apps\splunk_app_microsoft_exchange\default\savedsearches.conf, line 1131: Cannot parse into key-value pair: | dedup Host | table Host | sort Host | outputlookup windows_hostmon_system
04-01-2015 16:44:02.415 +0800 INFO TailingProcessor - Could not send data to output queue (parsingQueue), retrying...
04-01-2015 16:44:02.789 +0800 WARN AuthorizationManager - Unknown role 'windows-admin'
04-01-2015 16:54:08.424 +0800 WARN TimeoutHeap - Either time adjusted forwards by, or event loop was descheduled for 612234ms.
04-01-2015 17:13:08.842 +0800 INFO TcpInputProc - Stopping IPv4 port 514
04-01-2015 17:13:08.842 +0800 INFO TcpInputProc - Stopping IPv4 port 9997
04-01-2015 17:13:08.842 +0800 WARN TcpInputProc - Stopping all listening ports. Queues blocked for more than 300 seconds

---

2015-04-01 14:38:35,471 INFO [551b8ca6002385e10] root:129 - ENGINE: Console event 1: shutting down bus
2015-04-01 14:38:35,471 INFO [551b8ca6002385e10] root:129 - ENGINE: Removed handler for console events.
2015-04-01 14:38:35,471 INFO [551b8ca6002385e10] root:129 - ENGINE: Bus STOPPING
2015-04-01 14:38:36,252 INFO [551b8ca6002385e10] root:129 - ENGINE: HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 8065)) shut down
2015-04-01 14:38:36,266 INFO [551b8ca6002385e10] root:129 - ENGINE: Stopped thread 'Monitor'.
2015-04-01 14:38:36,329 INFO [551b8ca6002385e10] root:129 - ENGINE: Stopped thread 'Monitor'.
2015-04-01 14:38:36,329 INFO [551b8ca6002385e10] root:129 - ENGINE: Stopped thread '_TimeoutMonitor'.
2015-04-01 14:38:36,329 ERROR [551b8ca6002385e10] root:129 - ENGINE: Handler for console events already off.
2015-04-01 14:38:36,329 INFO [551b8ca6002385e10] root:129 - ENGINE: Bus STOPPED
2015-04-01 14:38:36,329 INFO [551b8ca6002385e10] root:129 - ENGINE: Bus EXITING
2015-04-01 14:38:36,329 INFO [551b8ca6002385e10] root:129 - ENGINE: Bus EXITED
2015-04-01 14:38:36,329 INFO [551b8ca6002385e10] root:129 - ENGINE: Waiting for child threads to terminate...

[sql_a]

As it turns out - after wasting hours on trying to figure out why the web interface on the index peers didn't start anymore - I figured out that after the index peer as been successfully connected to a cluster master - it will not start the web interface for the peer (even though port 8000 is reported as open) unless the Index Master is started first.
I'm not sure at which point this changed, because the peers have been restarted successfully several times until now...

Short answer: Start any Cluster Master before starting any peer - that may solve your problem (at least in 6.4.3)

/A

[yannK]

Use a btool command web.conf if you are using the port 8000

splunk.exe cmd btool web list 

Check if no other processes have claimed the port 8000.
to look at the port per process and pid - windows

netstat -b -a -o | find "8000"

[jackywsy]

Hi yannk, I execute first command ,just a line httpport =8000 listed,
second command ,nothing was listed ,

I thought I use the default settings and I didn't change anything. Just install some APPs before. Thanks for your response.

[masonmorales]

Have you tried:

https://localhost:8000 

and

https://localhost:8089

[jackywsy]

Hi Masonmorales,

I tried https://localhost:8000 ,it is the same as http://localhost:8000

Https://localhost:8089 is working well .And I can login into the management items .What can I do next? Thanks

[mthierbel]

I'm having the exact same problem with 6.1.3. and Windows 7. Did you ever find a solution? I have yet to.

[jackywsy]

It said the webpage is not available.