I am running Splunk Add-on for Tenable 5.1.1 on a heavy forwarder that is running Splunk 6.6.3 to pull vulnerability data from Tenable Security Center. I am having an issue where the add-on stops pulling. This usually happens around once a day. The odd the is I am not getting any errors or even warnings in the logs. It will be working perfectly and then just stops pulling. When it stops I can go in, disable the input, change the checkpoint value to the timestamp of the last event pulled in, then re-enable the input and it will start pulling in again without issue.
I can see in the logs that it is actually still attempting to do something even though it isn't pulling anything. The following is getting logged every 2 minutes (which is the schedule it runs on):
2017-09-11 20:06:57,601 +0000 log_level=INFO, pid=13810, tid=Thread-2, file=scheduler.py, func_name=get_ready_jobs, code_line_no=100 | Get 1 ready jobs, next duration is 119.999509, and there are 1 jobs scheduling
2017-09-11 20:06:57,601 +0000 log_level=INFO, pid=13810, tid=Thread-6, file=thread_pool.py, func_name=_run, code_line_no=261 | Thread work_queue_size=0
Any ideas would be greatly appreciated.
Note: I have been dealing with this for quite a while. Even when the HF was on Splunk 6.5.x.
... View more