Monitoring Splunk

My splunkd crashing very frequently

dhs_harry08
Path Finder

Hi,
My splunk is crashing every few days.
Below are the error logs that I see. Has anyone experienced this issue before. I cant figure out the traceback. Can someone pleas explain me what the below log means and how to troubleshoot the issue.

2013-11-10 07:31:55,935 ERROR   [527f285bed1ec4310] startup:84 - Unable to read in product version information; Splunkd daemon is not responding: ('[Errno 111] Connection refused',)

2013-11-10 07:31:55,936 ERROR   [527f285bed1ec4310] decorators:383 - Splunkd daemon is not responding: ('[Errno 111] Connection refused',)
Traceback (most recent call last):
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 369, in handle_exceptions
    return fn(self, *a, **kw)
  File "<string>", line 1, in <lambda>
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 424, in apply_cache_headers
response = fn(self, *a, **kw)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/controllers/account.py", line 211, in login
    sessionKey = splunk.auth.getSessionKey(username, password, hostPath=self.splunkd_urlhost)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/auth.py", line 28, in getSessionKey
serverResponse, serverContent = rest.simpleRequest(uri, postargs=args)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/rest/__init__.py", line 443, in simpleRequest
    raise splunk.SplunkdConnectionException, str(e)
SplunkdConnectionException: Splunkd daemon is not responding: ('[Errno 111] Connection refused',)
Tags (2)
0 Karma
1 Solution

dhs_harry08
Path Finder

I found the issue. It was the disk space issue. As I was using LVM the disk space was not properly showing up.

View solution in original post

0 Karma

dhs_harry08
Path Finder

I found the issue. It was the disk space issue. As I was using LVM the disk space was not properly showing up.

0 Karma
Get Updates on the Splunk Community!

Accelerating Observability as Code with the Splunk AI Assistant

We’ve seen in previous posts what Observability as Code (OaC) is and how it’s now essential for managing ...

Integrating Splunk Search API and Quarto to Create Reproducible Investigation ...

 Splunk is More Than Just the Web Console For Digital Forensics and Incident Response (DFIR) practitioners, ...

Congratulations to the 2025-2026 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...