The Splunk instance that I work with has several data input scripts. (~30) One of them is scheduled to run hourly, it's cron string looks like this: "0 * * * *" It was working great, but it abruptly stopped running for some reason. The last time it ran, (as determined by the timestamp on it's output files) was June 30th at 11:00pm. I restarted splunk, and it started running again, but I still haven't been able to determine what caused it to stop in the first place.
Because of the timing, I thought it might be an error in my cron string, but everything I have found online says that the string is correct. Has anyone else run into this before? Scripts that abruptly stop running?
I am seeing the same issue. It will just randomly get removed from the schedule. Debug refresh will cause it to get added to the schedule again, but it will not start running again until I disable and re-enable. I have checked the _internal and opened up the splunkd.log locally. No errors. I have performed the basic troubleshooting listed by grijhwani and no luck.
Do you understand the script, or are you simply relying on a legacy job instated by someone else? You give no indication as to what the script is, or your underlying platform, so it is hard to answer the question. A number of generic possibilities:
In short - have you done any debugging before assuming it is the scheduling which has broken?
You could find out more by looking in the Splunk logs. Log into Splunk as a admin, and run this search
You should be able to see what happened each time that Splunk attempted to run your script.
You can find this same information in