Monitoring Splunk

My splunkd crashing very frequently

dhs_harry08
Path Finder

Hi,
My splunk is crashing every few days.
Below are the error logs that I see. Has anyone experienced this issue before. I cant figure out the traceback. Can someone pleas explain me what the below log means and how to troubleshoot the issue.

2013-11-10 07:31:55,935 ERROR   [527f285bed1ec4310] startup:84 - Unable to read in product version information; Splunkd daemon is not responding: ('[Errno 111] Connection refused',)

2013-11-10 07:31:55,936 ERROR   [527f285bed1ec4310] decorators:383 - Splunkd daemon is not responding: ('[Errno 111] Connection refused',)
Traceback (most recent call last):
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 369, in handle_exceptions
    return fn(self, *a, **kw)
  File "<string>", line 1, in <lambda>
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 424, in apply_cache_headers
response = fn(self, *a, **kw)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/controllers/account.py", line 211, in login
    sessionKey = splunk.auth.getSessionKey(username, password, hostPath=self.splunkd_urlhost)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/auth.py", line 28, in getSessionKey
serverResponse, serverContent = rest.simpleRequest(uri, postargs=args)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/rest/__init__.py", line 443, in simpleRequest
    raise splunk.SplunkdConnectionException, str(e)
SplunkdConnectionException: Splunkd daemon is not responding: ('[Errno 111] Connection refused',)
Tags (2)
0 Karma
1 Solution

dhs_harry08
Path Finder

I found the issue. It was the disk space issue. As I was using LVM the disk space was not properly showing up.

View solution in original post

0 Karma

dhs_harry08
Path Finder

I found the issue. It was the disk space issue. As I was using LVM the disk space was not properly showing up.

0 Karma
Get Updates on the Splunk Community!

Splunk AI Assistant for SPL | Key Use Cases to Unlock the Power of SPL

Splunk AI Assistant for SPL | Key Use Cases to Unlock the Power of SPL  The Splunk AI Assistant for SPL ...

Buttercup Games: Further Dashboarding Techniques (Part 5)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...

Customers Increasingly Choose Splunk for Observability

For the second year in a row, Splunk was recognized as a Leader in the 2024 Gartner® Magic Quadrant™ for ...