Monitoring Splunk

My splunkd crashing very frequently

dhs_harry08
Path Finder

Hi,
My splunk is crashing every few days.
Below are the error logs that I see. Has anyone experienced this issue before. I cant figure out the traceback. Can someone pleas explain me what the below log means and how to troubleshoot the issue.

2013-11-10 07:31:55,935 ERROR   [527f285bed1ec4310] startup:84 - Unable to read in product version information; Splunkd daemon is not responding: ('[Errno 111] Connection refused',)

2013-11-10 07:31:55,936 ERROR   [527f285bed1ec4310] decorators:383 - Splunkd daemon is not responding: ('[Errno 111] Connection refused',)
Traceback (most recent call last):
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 369, in handle_exceptions
    return fn(self, *a, **kw)
  File "<string>", line 1, in <lambda>
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 424, in apply_cache_headers
response = fn(self, *a, **kw)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/controllers/account.py", line 211, in login
    sessionKey = splunk.auth.getSessionKey(username, password, hostPath=self.splunkd_urlhost)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/auth.py", line 28, in getSessionKey
serverResponse, serverContent = rest.simpleRequest(uri, postargs=args)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/rest/__init__.py", line 443, in simpleRequest
    raise splunk.SplunkdConnectionException, str(e)
SplunkdConnectionException: Splunkd daemon is not responding: ('[Errno 111] Connection refused',)
Tags (2)
0 Karma
1 Solution

dhs_harry08
Path Finder

I found the issue. It was the disk space issue. As I was using LVM the disk space was not properly showing up.

View solution in original post

0 Karma

dhs_harry08
Path Finder

I found the issue. It was the disk space issue. As I was using LVM the disk space was not properly showing up.

0 Karma
Get Updates on the Splunk Community!

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Splunk Education Goes to Washington | Splunk GovSummit 2024

If you’re in the Washington, D.C. area, this is your opportunity to take your career and Splunk skills to the ...