Security

Unable to connect to the splunk web interface

cebo_myeza
Path Finder

I have been using the splunk web interface with this address 127.0.0.1:8000 for almost two months now, but all of the sudden it just stopped connecting and it's giving me a message saying that "Firefox can't establish a connection to the server at 127.0.0.1:8000".

please help i don't know what went wrong.

martin_mueller
SplunkTrust
SplunkTrust

It's really bad practice to run splunk as root.

cebo_myeza
Path Finder

hi martin_mueller

i always here this but i dont know why is not a good practice.

can you please explain to me why is not a good practice in simple terms

thank in advance

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Any piece of software - Splunk or not - should not be run as root unless there is a very good reason to do so.

n00badmin
Communicator

cd /opt/splunk/bin
./splunk stop

do you get a reponse??

0 Karma

cebo_myeza
Path Finder

hi n00badmin

From the look of things my system is healthy thou:

Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_wisdom-lv_root
50G 3.8G 45G 8% /
tmpfs 1.9G 376K 1.9G 1% /dev/shm
/dev/sda1 477M 98M 354M 22% /boot
/dev/mapper/vg_wisdom-lv_home
176G 18G 149G 11% /home

0 Karma

n00badmin
Communicator

so strange...

is this a licenced deployment or are you running the free licence?

0 Karma

cebo_myeza
Path Finder

i am running a free licence for now...

0 Karma

n00badmin
Communicator

at this point it is worth gathering your data and re-installing????

0 Karma

ngatchasandra
Builder

Hi cebo_myeza,

  • Try to check if the attribute startwebserver of web.conf file is set to 1.

    web.conf is located to $SPLUNK_HOME/etc/system/default/

  • Or try to run individualy the command that follow:

    sudo /opt/splunk/bin/splunkd start and sudo /opt/splunk/bin/splunkweb start

cebo_myeza
Path Finder

if i check the attributes of startwebserver everything seems okay here is the copy below:

[settings]

enable/disable the appserver

startwebserver = 1

httpport = 8000

enableSplunkWebSSL = false

mgmtHostPort = 127.0.0.1:8089

appServerPorts = 8065

And if i run the command 'sudo /opt/splunk/bin/splunkd start' i get the following error

[root@localhost wisdom.network_trainee]# sudo /opt/splunk/bin/splunkweb start
sudo: /opt/splunk/bin/splunkweb: command not found

[root@localhost wisdom.network_trainee]# sudo /opt/splunk/bin/splunkd start
/opt/splunk/bin/splunkd: error while loading shared libraries: libjemalloc.so.1: cannot open shared object file: No such file or directory

thanks for your time ngatchasandra

0 Karma

n00badmin
Communicator

dude,

you need to locate your install...

does /opt/splunk/bin exist?

if it does do:

cd /opt/splunk/bin
./splunk stop

what do you get???

0 Karma

cebo_myeza
Path Finder

i still don't get any response like this:

[root@localhost wisdom.network_trainee]# cd /opt/splunk/bin
[root@localhost bin]# ./splunk stop
[root@localhost bin]# ./splunk start
[root@localhost bin]#

0 Karma

n00badmin
Communicator

please do ps -ef | grep splunk

0 Karma

cebo_myeza
Path Finder

[root@localhost wisdom.network_trainee]# ps -ef | grep splunk
root 6417 6407 0 08:59 pts/1 00:00:00 grep splunk

0 Karma

n00badmin
Communicator

splunk is not running....

were you the one who installed splunk??

can you confirm that splunk is installed at /opt??

0 Karma

cebo_myeza
Path Finder

Yeah i am the one who installed splunk and i have been using the splunkweb for the past 3 months, everything was running smoothly, i really don't know what went wrong.

how can i confirm?

0 Karma

cebo_myeza
Path Finder

[root@localhost wisdom.network_trainee]# find / -name splunk

/etc/rc.d/init.d/splunk
/opt/splunk
/opt/splunk/lib/python2.7/site-packages/splunk
/opt/splunk/share/splunk
/opt/splunk/share/splunk/search_mrsparkle/exposed/js/splunk
/opt/splunk/share/splunk/search_mrsparkle/exposed/img/splunk
/opt/splunk/bin/splunk
/opt/splunk/var/log/splunk
/opt/splunk/var/run/splunk
/opt/splunk/var/lib/splunk
/opt/splunk/var/spool/splunk
/var/lock/subsys/splunk
/var/spool/mail/splunk

0 Karma

MuS
SplunkTrust
SplunkTrust

To me this looks like your /opt/splunk is almost empty!

There should be a hole lot more files in /opt/splunk like:

find /opt/splunk/ | wc -l
12581

or

du -sk /opt/splunk/
1385792 /opt/splunk/

cebo_myeza
Path Finder

hi Mus

i find this :

[root@localhost wisdom.network_trainee]# find /opt/splunk/ | wc -l
12750

[root@localhost wisdom.network_trainee]# du -sk /opt/splunk/
1575340 /opt/splunk/

0 Karma

cebo_myeza
Path Finder

i get this

[root@localhost wisdom.network_trainee]# strace /opt/splunk/bin/splunk start

execve("/opt/splunk/bin/splunk", ["/opt/splunk/bin/splunk", "start"], [/* 39 vars */]) = -1 ENOEXEC (Exec format error)
dup(2) = 3
fcntl(3, F_GETFL) = 0x8002 (flags O_RDWR|O_LARGEFILE)
fstat(3, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9bc4c17000
lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
write(3, "strace: exec: Exec format error\n", 32strace: exec: Exec format error
) = 32
close(3) = 0
munmap(0x7f9bc4c17000, 4096) = 0
exit_group(1) = ?

Thanks

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...