topic Re: Unable to connect to the splunk web interface in Security

Unable to connect to the splunk web interface

cebo_myeza — Tue, 23 Jun 2015 02:25:47 GMT

I have been using the splunk web interface with this address 127.0.0.1:8000 for almost two months now, but all of the sudden it just stopped connecting and it's giving me a message saying that "Firefox can't establish a connection to the server at 127.0.0.1:8000".

please help i don't know what went wrong.

Re: Unable to connect to the splunk web interface

n00badmin — Tue, 23 Jun 2015 03:10:30 GMT

is splunk running?

what OS are you on?

Re: Unable to connect to the splunk web interface

martin_mueller — Tue, 23 Jun 2015 04:33:23 GMT

Assuming Splunk is running, someone may have changed the port or HTTPS encryption.

Re: Unable to connect to the splunk web interface

cebo_myeza — Tue, 23 Jun 2015 06:22:14 GMT

hi n00badmin

i am using centos and splunk is running

Re: Unable to connect to the splunk web interface

esix_splunk — Tue, 23 Jun 2015 06:29:42 GMT

Try restarting your instance of splunk : 'sudo /opt/splunk/bin/splunk restart'. Check the message when it restarts and confirm what IP and port Splunk is bound to. (It will say started on http://localhost:8000 or https://localhost:XXXX"

Check if iptables has been started or running : service iptables status service iptables stop

Re: Unable to connect to the splunk web interface

cebo_myeza — Tue, 23 Jun 2015 07:43:01 GMT

hi esix_splunk

The iptables seems ok but when i run the command 'sudo /opt/splunk/bin/splunk restart' i don't get any response.

i get this:

[root@...]# sudo /opt/splunk/bin/splunk restart
[root@..]#

Re: Unable to connect to the splunk web interface

ngatchasandra — Tue, 23 Jun 2015 08:13:05 GMT

Hi cebo_myeza,

Try to check if the attribute startwebserver of web.conf file is set to 1.

web.conf is located to $SPLUNK_HOME/etc/system/default/
Or try to run individualy the command that follow:

sudo /opt/splunk/bin/splunkd start and sudo /opt/splunk/bin/splunkweb start

Re: Unable to connect to the splunk web interface

n00badmin — Tue, 23 Jun 2015 11:12:51 GMT

cd /opt/splunk/bin
./splunk stop

do you get a reponse??

Re: Unable to connect to the splunk web interface

martin_mueller — Tue, 23 Jun 2015 18:58:36 GMT

It's really bad practice to run splunk as root.

Re: Unable to connect to the splunk web interface

cebo_myeza — Thu, 25 Jun 2015 01:30:31 GMT

if i check the attributes of startwebserver everything seems okay here is the copy below:

[settings]

enable/disable the appserver

startwebserver = 1

httpport = 8000

enableSplunkWebSSL = false

mgmtHostPort = 127.0.0.1:8089

appServerPorts = 8065

And if i run the command 'sudo /opt/splunk/bin/splunkd start' i get the following error

[root@localhost wisdom.network_trainee]# sudo /opt/splunk/bin/splunkweb start
sudo: /opt/splunk/bin/splunkweb: command not found

[root@localhost wisdom.network_trainee]# sudo /opt/splunk/bin/splunkd start
/opt/splunk/bin/splunkd: error while loading shared libraries: libjemalloc.so.1: cannot open shared object file: No such file or directory

thanks for your time ngatchasandra

Re: Unable to connect to the splunk web interface

n00badmin — Thu, 25 Jun 2015 01:45:32 GMT

dude,

you need to locate your install...

does /opt/splunk/bin exist?

if it does do:

cd /opt/splunk/bin
./splunk stop

what do you get???

Re: Unable to connect to the splunk web interface

cebo_myeza — Thu, 25 Jun 2015 01:49:59 GMT

i still don't get any response like this:

[root@localhost wisdom.network_trainee]# cd /opt/splunk/bin
[root@localhost bin]# ./splunk stop
[root@localhost bin]# ./splunk start
[root@localhost bin]#

Re: Unable to connect to the splunk web interface

n00badmin — Thu, 25 Jun 2015 01:53:48 GMT

please do ps -ef | grep splunk

Re: Unable to connect to the splunk web interface

cebo_myeza — Thu, 25 Jun 2015 01:58:50 GMT

[root@localhost wisdom.network_trainee]# ps -ef | grep splunk
root 6417 6407 0 08:59 pts/1 00:00:00 grep splunk

Re: Unable to connect to the splunk web interface

n00badmin — Thu, 25 Jun 2015 02:18:56 GMT

splunk is not running....

were you the one who installed splunk??

can you confirm that splunk is installed at /opt??

Re: Unable to connect to the splunk web interface

cebo_myeza — Thu, 25 Jun 2015 02:26:50 GMT

Yeah i am the one who installed splunk and i have been using the splunkweb for the past 3 months, everything was running smoothly, i really don't know what went wrong.

how can i confirm?

Re: Unable to connect to the splunk web interface

cebo_myeza — Mon, 28 Sep 2020 20:22:31 GMT

[root@localhost wisdom.network_trainee]# find / -name splunk

/etc/rc.d/init.d/splunk
/opt/splunk
/opt/splunk/lib/python2.7/site-packages/splunk
/opt/splunk/share/splunk
/opt/splunk/share/splunk/search_mrsparkle/exposed/js/splunk
/opt/splunk/share/splunk/search_mrsparkle/exposed/img/splunk
/opt/splunk/bin/splunk
/opt/splunk/var/log/splunk
/opt/splunk/var/run/splunk
/opt/splunk/var/lib/splunk
/opt/splunk/var/spool/splunk
/var/lock/subsys/splunk
/var/spool/mail/splunk

Re: Unable to connect to the splunk web interface

n00badmin — Thu, 25 Jun 2015 02:48:11 GMT

can you check your splunkd.log please

cd /opt/splunk/var/log/splunk/

tail -100 splunkd.log

dont understand why you get nothing calling ./splunk start or stop

Re: Unable to connect to the splunk web interface

cebo_myeza — Mon, 28 Sep 2020 20:22:34 GMT

[root@localhost wisdom.network_trainee]# cd /opt/splunk/var/log/splunk/

[root@localhost splunk]# tail -50 splunkd.log
06-18-2015 10:40:47.859 +0700 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/opt/splunk/var/lib/splunk/_internaldb/db'. Reason='Removing bucket, bid=_internal~26~414C0929-B95F-4C97-A4BB-AB1267F53365'
06-18-2015 10:40:47.890 +0700 INFO BucketMover - AsyncFreezer freeze succeeded for bkt='/opt/splunk/var/lib/splunk/_internaldb/db/db_1432006804_1431942980_26'
06-18-2015 10:40:48.839 +0700 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/opt/splunk/var/lib/splunk/_internaldb/db'. Reason=' frozen_buckets'
06-18-2015 10:41:44.726 +0700 INFO WatchedFile - Will begin reading at offset=0 for file='/opt/splunk/var/log/splunk/metrics.log'.
06-18-2015 10:41:44.864 +0700 INFO WatchedFile - Will begin reading at offset=24992205 for file='/opt/splunk/var/log/splunk/metrics.log.1'.
06-18-2015 10:42:48.840 +0700 INFO BucketMover - will attempt to freeze: candidate='/opt/splunk/var/lib/splunk/_internaldb/db/db_1432006922_1432006809_27' because frozenTimePeriodInSecs=2592000 exceeds difference between now=1434598968 and latest=1432006922
06-18-2015 10:42:48.843 +0700 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/opt/splunk/var/lib/splunk/_internaldb/db'. Reason='Removing bucket, bid=_internal~27~414C0929-B95F-4C97-A4BB-AB1267F53365'
06-18-2015 10:42:48.847 +0700 INFO BucketMover - AsyncFreezer freeze succeeded for bkt='/opt/splunk/var/lib/splunk/_internaldb/db/db_1432006922_1432006809_27'
06-18-2015 10:42:49.832 +0700 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/opt/splunk/var/lib/splunk/_internaldb/db'. Reason=' frozen_buckets'
06-18-2015 15:16:56.840 +0700 INFO BucketMover - will attempt to freeze: candidate='/opt/splunk/var/lib/splunk/_internaldb/db/db_1432023396_1432006926_28' because frozenTimePeriodInSecs=2592000 exceeds difference between now=1434615416 and latest=1432023396
06-18-2015 15:16:56.841 +0700 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/opt/splunk/var/lib/splunk/_internaldb/db'. Reason='Removing bucket, bid=_internal~28~414C0929-B95F-4C97-A4BB-AB1267F53365'
06-18-2015 15:16:56.867 +0700 INFO BucketMover - AsyncFreezer freeze succeeded for bkt='/opt/splunk/var/lib/splunk/_internaldb/db/db_1432023396_1432006926_28'
06-18-2015 15:16:57.832 +0700 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/opt/splunk/var/lib/splunk/_internaldb/db'. Reason=' frozen_buckets'
06-18-2015 17:03:47.839 +0700 INFO BucketMover - will attempt to freeze: candidate='/opt/splunk/var/lib/splunk/_introspection/db/db_1433412195_1433380011_38' because frozenTimePeriodInSecs=1209600 exceeds difference between now=1434621827 and latest=1433412195
06-18-2015 17:03:47.842 +0700 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/opt/splunk/var/lib/splunk/_introspection/db'. Reason='Removing bucket, bid=_introspection~38~414C0929-B95F-4C97-A4BB-AB1267F53365'
06-18-2015 17:03:47.862 +0700 INFO BucketMover - AsyncFreezer freeze succeeded for bkt='/opt/splunk/var/lib/splunk/_introspection/db/db_1433412195_1433380011_38'
06-18-2015 17:03:58.837 +0700 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/opt/splunk/var/lib/splunk/_introspection/db'. Reason=' frozen_buckets'
06-18-2015 17:08:36.835 +0700 INFO BucketMover - will attempt to freeze: candidate='/opt/splunk/var/lib/splunk/_internaldb/db/db_1432030087_1432023400_29' because frozenTimePeriodInSecs=2592000 exceeds difference between now=1434622116 and latest=1432030087
06-18-2015 17:08:36.835 +0700 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/opt/splunk/var/lib/splunk/_internaldb/db'. Reason='Removing bucket, bid=_internal~29~414C0929-B95F-4C97-A4BB-AB1267F53365'
06-18-2015 17:08:36.844 +0700 INFO BucketMover - AsyncFreezer freeze succeeded for bkt='/opt/splunk/var/lib/splunk/_internaldb/db/db_1432030087_1432023400_29'
06-18-2015 17:08:37.831 +0700 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/opt/splunk/var/lib/splunk/_internaldb/db'. Reason=' frozen_buckets'
06-19-2015 00:00:00.831 +0700 INFO LMStackMgr - should rollover=true because _lastRolloverTime=1434560400 lastRolloverDay=1434560400 snappedNow=1434646800
06-19-2015 00:00:00.832 +0700 INFO LMStackMgr - quotaExceededCount=0, lastExceedDate=0, peak=0, rolloverCount=1, totalCumulativeBytesAtRollover=0, todaysBytesIndexed=0, licenseSize=524288000
06-19-2015 00:00:00.832 +0700 INFO LMStackMgr - finished rollover, new lastRolloverTime=1434646800
06-19-2015 00:00:03.831 +0700 INFO LMSlaveInfo - Detected that masterTimeFromSlave(Thu Jun 18 23:59:02 2015) < lastRolloverTime(Fri Jun 19 00:00:00 2015), meaning that the master has already rolled over. Ignore slave persisted usage.
06-19-2015 07:36:27.586 +0700 INFO WatchedFile - Will begin reading at offset=0 for file='/opt/splunk/var/log/introspection/resource_usage.log'.
06-19-2015 07:36:27.668 +0700 INFO WatchedFile - Will begin reading at offset=24998813 for file='/opt/splunk/var/log/introspection/resource_usage.log.1'.
06-19-2015 09:24:41.898 +0700 INFO WatchedFile - Will begin reading at offset=0 for file='/opt/splunk/var/log/splunk/metrics.log'.
06-19-2015 09:24:42.020 +0700 INFO WatchedFile - Will begin reading at offset=24997732 for file='/opt/splunk/var/log/splunk/metrics.log.1'.
06-19-2015 10:42:14.568 +0700 ERROR HTTPClient - Cannot resolve IP of host=apps.splunk.com: Temporary failure in name resolution
06-19-2015 10:42:14.568 +0700 ERROR ApplicationUpdater - Error checking for update, URL=/api/apps:resolve/checkforupgrade: Invalid URI
06-19-2015 10:51:20.832 +0700 INFO LMStackMgr - should rollover=true because _lastRolloverTime=1434646800 lastRolloverDay=1434560400 snappedNow=1434646800
06-19-2015 10:51:20.832 +0700 INFO LMStackMgr - quotaExceededCount=0, lastExceedDate=0, peak=0, rolloverCount=2, totalCumulativeBytesAtRollover=0, todaysBytesIndexed=0, licenseSize=524288000
06-19-2015 10:51:20.832 +0700 INFO LMStackMgr - finished rollover, new lastRolloverTime=1434685880
06-19-2015 10:51:48.586 +0700 WARN TimeoutHeap - Detected system time adjusted backwards by 11245ms.
06-19-2015 10:51:48.663 +0700 WARN TimeoutHeap - Detected system time adjusted backwards by 11244ms.
06-19-2015 10:51:48.747 +0700 WARN TimeoutHeap - Detected system time adjusted backwards by 11244ms.
06-19-2015 10:51:48.765 +0700 WARN TimeoutHeap - Detected system time adjusted backwards by 2326ms.
06-19-2015 10:51:48.908 +0700 WARN IntrospectionGenerator:resource_usage - TimeoutHeap - Detected system time adjusted backwards by 7247ms.
06-19-2015 10:51:48.908 +0700 WARN TimeoutHeap - Detected system time adjusted backwards by 7247ms.
06-19-2015 10:51:52.849 +0700 WARN TimeoutHeap - Detected system time adjusted backwards by 2246ms.
06-19-2015 10:51:52.849 +0700 WARN TimeoutHeap - Detected system time adjusted backwards by 2246ms.
06-19-2015 10:52:16.584 +0700 INFO LMSlaveInfo - Detected that masterTimeFromSlave(Fri Jun 19 10:51:15 2015) < lastRolloverTime(Fri Jun 19 10:51:20 2015), meaning that the master has already rolled over. Ignore slave persisted usage.
06-19-2015 13:41:17.605 +0700 INFO BucketMover - will attempt to freeze: candidate='/opt/splunk/var/lib/splunk/_internaldb/db/db_1432104061_1432030091_30' because frozenTimePeriodInSecs=2592000 exceeds difference between now=1434696077 and latest=1432104061
06-19-2015 13:41:17.606 +0700 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/opt/splunk/var/lib/splunk/_internaldb/db'. Reason='Removing bucket, bid=_internal~30~414C0929-B95F-4C97-A4BB-AB1267F53365'
06-19-2015 13:41:17.629 +0700 INFO BucketMover - AsyncFreezer freeze succeeded for bkt='/opt/splunk/var/lib/splunk/_internaldb/db/db_1432104061_1432030091_30'
06-19-2015 13:41:18.585 +0700 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/opt/splunk/var/lib/splunk/_internaldb/db'. Reason=' frozen_buckets'
06-19-2015 16:50:44.599 +0700 INFO BucketMover - will attempt to freeze: candidate='/opt/splunk/var/lib/splunk/_introspection/db/db_1433497801_1433466532_39' because frozenTimePeriodInSecs=1209600 exceeds difference between now=1434707444 and latest=1433497801
06-19-2015 16:50:44.605 +0700 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/opt/splunk/var/lib/splunk/_introspection/db'. Reason='Removing bucket, bid=_introspection~39~414C0929-B95F-4C97-A4BB-AB1267F53365'
06-19-2015 16:50:44.608 +0700 INFO BucketMover - AsyncFreezer freeze succeeded for bkt='/opt/splunk/var/lib/splunk/_introspection/db/db_1433497801_1433466532_39'

Re: Unable to connect to the splunk web interface

n00badmin — Thu, 25 Jun 2015 03:12:53 GMT

i doubt u were able to post the entire 100 lines of log...

do a tail -10 please...whats ur last 10 events in the log?