Hi, My splunk is crashing every few days. Below are the error logs that I see. Has anyone experienced this issue before. I cant figure out the traceback. Can someone pleas explain me what the below log means and how to troubleshoot the issue. 2013-11-10 07:31:55,935 ERROR [527f285bed1ec4310] startup:84 - Unable to read in product version information; Splunkd daemon is not responding: ('[Errno 111] Connection refused',) 2013-11-10 07:31:55,936 ERROR [527f285bed1ec4310] decorators:383 - Splunkd daemon is not responding: ('[Errno 111] Connection refused',) Traceback (most recent call last): File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 369, in handle_exceptions return fn(self, *a, **kw) File "<string>", line 1, in <lambda> File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 424, in apply_cache_headers response = fn(self, *a, **kw) File "/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle/controllers/account.py", line 211, in login sessionKey = splunk.auth.getSessionKey(username, password, hostPath=self.splunkd_urlhost) File "/opt/splunk/lib/python2.7/site-packages/splunk/auth.py", line 28, in getSessionKey serverResponse, serverContent = rest.simpleRequest(uri, postargs=args) File "/opt/splunk/lib/python2.7/site-packages/splunk/rest/__init__.py", line 443, in simpleRequest raise splunk.SplunkdConnectionException, str(e) SplunkdConnectionException: Splunkd daemon is not responding: ('[Errno 111] Connection refused',) ... View more

I have a default inputs.conf configuration in splunk server as below. [batch:///source=/opt/splunk/var/spool/splunk/] move_policy = sinkhole Can I disable it will it have any adverse affect to my splunk server. The thing is there are some sample logs coming into this directory and we are unable find the script or the guy who is doing it. Also can I put disable=true in this stanza will it work or should I comment the entire thing. Regards, Harish ... View more

Hi, I have scheduled a cron in unix for every one minute which takes the uptime command output and writes to a file. It has been over 4 weeks now and I am observing now that every day splunk timestamp changes abruptly at 16:29. You can see from below logs the event after 2013-05-27T16:29:00.000+0530 is coming as 2013-05-28T04:30:00.000+0530 and because of this a lot of events are missing and not getting reported. 1:00pm up 146 days 1:10, 1 user, load average: 2.11, 2.27, 3.20 2013-05-28T04:30:00.000+0530 1 28 0 may tuesday 2013 local nix-all-logs byk300lin241 3.2 2.11 2.27 /home/tkaushal/load.log unix_load 6 2 12:59am up 145 days 13:09, 0 users, load average: 0.24, 0.84, 1.15 2013-05-27T16:29:00.000+0530 12 27 59 may monday 2013 local nix-all-logs byk300lin241 1.15 0.24 0.84 /home/tkaushal/load.log unix_load 6 1 Please help me understand why is this ocuring and solution for it. Regards, Harish ... View more

hi, I have this source showing in the splunk source=/opt/splunk/var/spool/splunk/singlehost.sample.sav But when I checked the server I am unable to find this particular file. How is this possible. If I want to find the source in this case how can I do it. For example if the data is coming through a network port. Regards, Harish ... View more

Hi, I have a requirement wherein I am using bucket to calculate range and their values. host="hobbes8" |search success OR fail|eventstats count(txnId) as gtotal |bucket api_time bins=100|eventstats count(txnId) as total by api_time|eval percent=100*total/gtotal |table api_time,percent,total. I am getting my result but im getting many dupliicate entries. An when I am using stats I am getting the error. api_time should be numerical. Can you help me with this. I need the result like this. api_time percent total 1 1500-1600 0.277316 10 2 400-500 6.211869 224 3 800-900 2.329451 84 4 300-400 6.267332 226 Regards, Harish ... View more

Hi I was testing splunk by adding multiple servers by both syslog and splunk forwarder. But now my splunk indexer has all kinds of stuff that I don't need anymore. I want to remove some hosts, source and source types and the data it contains. Per the document I should be able to delete by removing stanzas in indexes.conf. But i can't find the entries in it. Please help me out. Regards, Harish ... View more