Installation

Why is Splunk Search head is not getting restarted?

sivakrishna
Path Finder

Hi Team,

I am unable to restart Splunk on my Search head. Earlier its working fine but now it's not working.

Due to this GUI of this Search head is also not able to access getting timed out.

sivakrishna_1-1690182284254.png

It's getting stopped here and no response I'm getting. I kept under observation up to 30min but no response.

web.conf

sivakrishna_2-1690182516273.png

 

Labels (1)
Tags (1)
0 Karma
1 Solution

isoutamo
SplunkTrust
SplunkTrust
If it’s a new installation without any real data, then probably the easiest solution is remove it and then reinstall it?

View solution in original post

0 Karma

sivakrishna
Path Finder

Getting this Splunkd.log

 

07-24-2023 09:54:21.053 +0100 ERROR BTreeCP [2041288 indexerPipe] - checkpoint failed: removal of dir %s /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/snapshot.old: 1 errors occurred. Description for first 1: [{operation:"failed to remove directory", error:"Directory not empty", file:"/opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/snapshot.old"}]

0 Karma

jotne
Builder

Have you done any changes to the search head.  Use GIT or some other to track your changes if you do not use that.

0 Karma

sivakrishna
Path Finder

No Its newly installed Splunk server

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

have you anything in /opt/splunk/var/log/splunk/splunkd.log which could explain the reason why http service hasn’t started?

r. Ismo

0 Karma

sivakrishna
Path Finder

Getting this Splunkd.log

 

07-24-2023 09:54:21.053 +0100 ERROR BTreeCP [2041288 indexerPipe] - checkpoint failed: removal of dir. %s /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/snapshot.old: 1 error occurred. Description for first 1: [{operation:"failed to remove directory", error:"Directory not empty", file:"/opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/snapshot.old"}]

0 Karma

isoutamo
SplunkTrust
SplunkTrust
If it’s a new installation without any real data, then probably the easiest solution is remove it and then reinstall it?
0 Karma

sivakrishna
Path Finder

I tried the same thing as you suggested then Issue got Resolved.

Thank you!!!

0 Karma
Get Updates on the Splunk Community!

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  🚀 Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Accelerating Observability as Code with the Splunk AI Assistant

We’ve seen in previous posts what Observability as Code (OaC) is and how it’s now essential for managing ...