Getting Data In

Splunk indexer service: Why error "RHEL 7.1 systemd[1]: Failed to start SYSV: Splunk indexer service"?

lraynal
Explorer

My Splunk indexer is not starting as a service on RHEL 7.1 on a fresh install.
It's starting ok as splunk user though.

 [root@myindexer ~]# systemctl status splunk
    splunk.service - SYSV: Splunk indexer service
       Loaded: loaded (/etc/rc.d/init.d/splunk)
       Active: failed (Result: exit-code) since mer. 2015-09-30 18:21:15 CEST; 4min 13s ago

    sept. 30 18:21:15 myindexer splunk[2938]: Starting Splunk...
    sept. 30 18:21:15 myindexer splunk[2938]: Splunk> Needle. Haystack. Found.
    sept. 30 18:21:15 myindexer splunk[2938]: Checking prerequisites...
    sept. 30 18:21:15 myindexer splunk[2938]: Checking http port [443]: already bound
    sept. 30 18:21:15 myindexer splunk[2938]: ERROR: The http port [443] is already bound.  Splunk needs to use this port.
    sept. 30 18:21:15 myindexer splunk[2938]: Would you like to change ports? [y/n]:
    sept. 30 18:21:15 myindexer splunk[2938]: Exiting due to --no-prompt.
    sept. 30 18:21:15 myindexer systemd[1]: splunk.service: control process exited, code=exited status=1
    sept. 30 18:21:15 myindexer systemd[1]: Failed to start SYSV: Splunk indexer service.
    sept. 30 18:21:15 myindexer systemd[1]: Unit splunk.service entered failed state.

Previously I did change Splunk Web server port to HTTPS / 443

    # echo "/opt/splunk/lib" > /etc/ld.so.conf.d/splunk.x86_64.conf
    # ldconfig
    # setcap 'cap_net_bind_service=+ep' /opt/splunk/bin/splunk
    # setcap 'cap_net_bind_service=+ep' /opt/splunk/bin/splunkd
    # su - splunk
    $ splunk --accept-license edit user admin -password $SPLUNK_PASSWORD -auth admin:changeme
    $ splunk set web-port 443
[..]
Labels (1)
0 Karma
1 Solution

lraynal
Explorer

This is in fact a problem with /opt/splunk/bin/splunk enable boot-start -user splunk
which installs a /etc/init.d/splunk that does everything as root, not splunk.

I added su splunk -c everywhere it's launching splunk, as in
su splunk -c "/opt/splunk/bin/splunk start --no-prompt --answer-yes"

View solution in original post

0 Karma

aasraoui
Loves-to-Learn

Hi,

would like to know where i can modify splunk db variable to point to a new directory with larger storage capacity.  

 

thanks

abdelillah

0 Karma

gbedsaul1
New Member

I'm getting a similar error to this, but I have no idea where it might be:

"""
[root@forwarder /opt/splunk]# systemctl -l status splunk
● splunk.service
Loaded: not-found (Reason: No such file or directory)
Active: failed (Result: exit-code) since Wed 2019-09-04 06:48:01 UTC; 49min ago

Sep 04 06:48:01 myforwarder splunk[4819]: and do not create a new session
Sep 04 06:48:01 myforwarder splunk[4819]: -f, --fast pass -f to the shell (for csh or tcsh)
Sep 04 06:48:01 myforwarder splunk[4819]: -s, --shell run shell if /etc/shells allows it
Sep 04 06:48:01 myforwarder splunk[4819]: -h, --help display this help and exit
Sep 04 06:48:01 myforwarder splunk[4819]: -V, --version output version information and exit
Sep 04 06:48:01 myforwarder splunk[4819]: For more details see su(1).
Sep 04 06:48:01 myforwarder systemd[1]: splunk.service: control process exited, code=exited status=1
Sep 04 06:48:01 myforwarder systemd[1]: Failed to start SYSV: Splunk indexer service.
Sep 04 06:48:01 myforwarder systemd[1]: Unit splunk.service entered failed state.
Sep 04 06:48:01 myforwarder systemd[1]: splunk.service failed.
"""

Especially since it's supposed to be running as a forwarder... Oy

0 Karma

buntel
New Member

I did the following and it worked. Don't ask me why since I am not an expert 😄
sudo chown -R splunk:splunk /opt/splunk

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

The why on this is that you gave the splunk userid the ownership of all files in the /opt/splunk directory, and recursively (-R) below that. So that error was a file permissions issue for you.

0 Karma

lraynal
Explorer

This is in fact a problem with /opt/splunk/bin/splunk enable boot-start -user splunk
which installs a /etc/init.d/splunk that does everything as root, not splunk.

I added su splunk -c everywhere it's launching splunk, as in
su splunk -c "/opt/splunk/bin/splunk start --no-prompt --answer-yes"

0 Karma

woodcock
Esteemed Legend

Google "splunk user bob docs". It is a sad situation that Splunk the enable boot-start command does not have an option for this.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...