Deployment Architecture

Splunk not restart

arun_kant_sharm
Path Finder

alt textHi experts,
I try to restart our splunk server, but its not start.

Earlier I try to start from UI, but it not start.
I also try to reboot if using CLI, but dont see any thing on console

I am using Splunk 7.2 in AWS EC2 instance (Amazon 1) , I am using splunk on that environment from last one year.

$SPLUNK_HOME/bin/splunk -version
$SPLUNK_HOME/bin/splunk -version
Splunk 7.2.6 (build c0bf0f679ce9)

uname -a
Linux abcdXyz 4.14.123-86.109.amzn1.x86_64 #1 SMP Mon Jun 10 19:44:53 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

strace /opt/splunk/bin/splunk start
execve("/opt/splunk/bin/splunk", ["/opt/splunk/bin/splunk", "start"], [/ 50 vars /]) = -1 ENOEXEC (Exec format error)
write(2, "strace: exec: Exec format error\n", 32strace: exec: Exec format error
) = 32
exit_group(1) = ?
+++ exited with 1 +++

Tags (1)
0 Karma

ayush1906
Path Finder

Hi Arun,

You are logged in as root user, and that does not have access to restart splunk.
Either do -> sudo su - splunk , then give restart command

or use chown command to change the owner to splunk then it will surely work.

Kindly accept this as answer if it works for you 🙂

0 Karma

arun_kant_sharm
Path Finder

ll splunk*
-r-xr-xr-x 1 splunk splunk 0 May 21 04:13 splunk
-r-xr-xr-x 1 splunk splunk 49356952 Apr 11 2019 splunkd
-r-xr-xr-x 1 splunk splunk 465 Apr 11 2019 splunkdj
-r-xr-xr-x 1 splunk splunk 21904 Apr 11 2019 splunkmon
-r-xr-xr-x 1 splunk splunk 295008 Apr 11 2019 splunk-optimize
-r-xr-xr-x 1 splunk splunk 291136 Apr 11 2019 splunk-optimize-lex

I don't know why my env splunk binary deleted, I only try to restart from UI. After replacing it from the other env, its working fine.

0 Karma

PavelP
Motivator

Hello @arun_kant_sharma

please try prepend strace to see more

strace /opt/splunk/bin/splunk start
0 Karma

arun_kant_sharm
Path Finder

strace /opt/splunk/bin/splunk start
execve("/opt/splunk/bin/splunk", ["/opt/splunk/bin/splunk", "start"], [/* 50 vars */]) = -1 ENOEXEC (Exec format error)
write(2, "strace: exec: Exec format error\n", 32strace: exec: Exec format error
) = 32
exit_group(1) = ?
+++ exited with 1 +++

0 Karma

PavelP
Motivator

@arun_kant_sharma this error means your computer architecture is different than the splunk binary

What is your OS (uname -a, lsb_release) and what is the exact splunk version (x64, 86, arm)?

0 Karma

arun_kant_sharm
Path Finder

I am using Splunk 7.2 in AWS EC2 instance (Amazon 1) , I am using splunk on that environment from last one year.

$SPLUNK_HOME/bin/splunk -version
Splunk 7.2.6 (build c0bf0f679ce9)

uname -a
Linux abcdXyz 4.14.123-86.109.amzn1.x86_64 #1 SMP Mon Jun 10 19:44:53 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

0 Karma

PavelP
Motivator

Has splunk suddently stopped to work or it happened after an upgrade?

please try

file /opt/splunk/bin/splunk*

expected output:

[root@mwg42 ~]# file /opt/splunk/bin/splunk*
/opt/splunk/bin/splunk:              ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.4.0, stripped
/opt/splunk/bin/splunkd:             ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.4.0, stripped
/opt/splunk/bin/splunkmon:           ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.4.0, stripped
/opt/splunk/bin/splunk-optimize:     ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.4.0, stripped
/opt/splunk/bin/splunk-optimize-lex: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.4.0, stripped
0 Karma

renjith_nair
Legend

@arun_kant_sharma ,

Its quite strange that you dont see anything in the console after the start command. Is the installation dir correct and are you able to see binaries there ?

---
What goes around comes around. If it helps, hit it with Karma 🙂
0 Karma

arun_kant_sharm
Path Finder

Yes Binaries are present in /opt/splunk/bin.

0 Karma

renjith_nair
Legend

@arun_kant_sharma ,
its possible that the binaries are overwritten by manual copy/move process. Otherwise it should output the start up messages in your console

---
What goes around comes around. If it helps, hit it with Karma 🙂
0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...