Why do I get "Splunk is not running" trying to run "list monitor" on my forwarder, and I can't stop splunk with error "Could not kill pid 379. [FAILED]"?

[efrenette11]

I can't list monitor my forwarder. I obtain "splunk is not running" even if the splunk status says it is running.

So I've tried to stop/start, but I can't do this either as I receive (Could not kill pid 379)

Any help ?

[splunk@app2.cort.jetdev2.syseng.tmcs ~]$ /opt/splunk/bin/splunk status
splunkd is running (PID: 379).
splunk helpers are running (PIDs: 381).
splunkweb is running (PID: 3091).
[splunk@app2.cort.jetdev2.syseng.tmcs ~]$ /opt/splunk/bin/splunk list monitor
Splunk is not running, and it must be for this operation. To start splunk, run "splunk start".
[splunk@app2.cort.jetdev2.syseng.tmcs ~]$ /opt/splunk/bin/splunk stop
                                                           [  OK  ]
Stopping splunkweb...Stopping splunkd...
Shutting down.  Please wait, as this may take a few minutes.
Could not kill pid 379.                                    [FAILED]

[efrenette11 · Accepted solution]

Ok I found this :

By removing the pid, it's working now.
splunkforwarder/var/run/splunk/splunkd.pid

[somesoni2]

Try something like this

 pkill splunk
 service splunk start

[efrenette11]

Tks for the help but this does not works. Any other's idea ?

[efrenette11]

Also, here's what I have a splunk process running.

[tmweb@app2.cort.jetdev2.syseng.tmcs ~]$ ps aux | grep splunk
tmweb     4044  0.0  0.0  61184   792 pts/0    S+   11:59   0:00 grep splunk
root      5969  0.0  0.0   8696  1200 ?        Ss   Aug16   0:00 /bin/bash -c sleep $(( $RANDOM * 86400 / 32767 )); [ -x /app/shared/bin/splunk_cfg_backup.sh ] && /app/shared/bin/splunk_cfg_backup.sh
[tmweb@app2.cort.jetdev2.syseng.tmcs ~]$