Monitoring Splunk

My splunkd crashing very frequently

dhs_harry08
Path Finder

Hi,
My splunk is crashing every few days.
Below are the error logs that I see. Has anyone experienced this issue before. I cant figure out the traceback. Can someone pleas explain me what the below log means and how to troubleshoot the issue.

2013-11-10 07:31:55,935 ERROR   [527f285bed1ec4310] startup:84 - Unable to read in product version information; Splunkd daemon is not responding: ('[Errno 111] Connection refused',)

2013-11-10 07:31:55,936 ERROR   [527f285bed1ec4310] decorators:383 - Splunkd daemon is not responding: ('[Errno 111] Connection refused',)
Traceback (most recent call last):
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 369, in handle_exceptions
    return fn(self, *a, **kw)
  File "<string>", line 1, in <lambda>
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 424, in apply_cache_headers
response = fn(self, *a, **kw)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/controllers/account.py", line 211, in login
    sessionKey = splunk.auth.getSessionKey(username, password, hostPath=self.splunkd_urlhost)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/auth.py", line 28, in getSessionKey
serverResponse, serverContent = rest.simpleRequest(uri, postargs=args)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/rest/__init__.py", line 443, in simpleRequest
    raise splunk.SplunkdConnectionException, str(e)
SplunkdConnectionException: Splunkd daemon is not responding: ('[Errno 111] Connection refused',)
Tags (2)
0 Karma
1 Solution

dhs_harry08
Path Finder

I found the issue. It was the disk space issue. As I was using LVM the disk space was not properly showing up.

View solution in original post

0 Karma

dhs_harry08
Path Finder

I found the issue. It was the disk space issue. As I was using LVM the disk space was not properly showing up.

0 Karma
Get Updates on the Splunk Community!

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...