Monitoring Splunk

My splunkd crashing very frequently

dhs_harry08
Path Finder

Hi,
My splunk is crashing every few days.
Below are the error logs that I see. Has anyone experienced this issue before. I cant figure out the traceback. Can someone pleas explain me what the below log means and how to troubleshoot the issue.

2013-11-10 07:31:55,935 ERROR   [527f285bed1ec4310] startup:84 - Unable to read in product version information; Splunkd daemon is not responding: ('[Errno 111] Connection refused',)

2013-11-10 07:31:55,936 ERROR   [527f285bed1ec4310] decorators:383 - Splunkd daemon is not responding: ('[Errno 111] Connection refused',)
Traceback (most recent call last):
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 369, in handle_exceptions
    return fn(self, *a, **kw)
  File "<string>", line 1, in <lambda>
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 424, in apply_cache_headers
response = fn(self, *a, **kw)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/controllers/account.py", line 211, in login
    sessionKey = splunk.auth.getSessionKey(username, password, hostPath=self.splunkd_urlhost)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/auth.py", line 28, in getSessionKey
serverResponse, serverContent = rest.simpleRequest(uri, postargs=args)
  File "/opt/splunk/lib/python2.7/site-packages/splunk/rest/__init__.py", line 443, in simpleRequest
    raise splunk.SplunkdConnectionException, str(e)
SplunkdConnectionException: Splunkd daemon is not responding: ('[Errno 111] Connection refused',)
Tags (2)
0 Karma
1 Solution

dhs_harry08
Path Finder

I found the issue. It was the disk space issue. As I was using LVM the disk space was not properly showing up.

View solution in original post

0 Karma

dhs_harry08
Path Finder

I found the issue. It was the disk space issue. As I was using LVM the disk space was not properly showing up.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...