Script Executing Late by the UF

[Priya70]

Hi, I have the following stanza in the inputs.conf:

[script://.\bin\test_data.path]
_TCP_ROUTING = test
disabled = false

interval = 0 11 * * *
index = testIndex
sourcetype = PowershellData

The script is being executed by the forwarder directly. The data is not being written to any file. The script executed at 12:24 Pm when its supposed to execute at 11 Am. I do have date/time line on the script that tracks when the script was executed. Why is the script executing late on the device. I am really confused. 

[dural_yyz]

1) How long does the script take to execute if you manually execute it on time (is something competing for resources)?

2) What log entries are available from the source to indicate what happens when script runs and or fails?

[Priya70]

@dural_yyz The script is very simple. If I manually run it, it gives results in 2 seconds. There were no errors in the logs. Also, I noticed that the device was offline at time Forwarder was supposed to execute the script (11 Am). The device came online around 12:24 Pm, which is when the data was sent. So, the question is, how come the script is being executed at 12:24 Pm when the execution time for the script was passed? Shouldn't it go to the next secluded execution right? It doing that on certain devices. 

[dural_yyz]

The spec file for input specifically calls out when cron is used the script should not run on start up.  That said you have evidence that it is.  I recommend deep diving on the splunkd.log files from the source to see how and possibly why the script was executed on startup.

$SPLUNK_HOME/var/log/splunk/splunkd.log

[Priya70]

@dural_yyz splunkd.log do not have information when the script was executed. I see no errors related to scripted input.

[dural_yyz]

At this point we are getting a bit beyond my personal experiences and now diving into my what if skill set.  The current logging would be out of the box defaults and may not be recording the event that tells you what is happening.  I would look into what debug levels you could increase but I don't know which specifically would help you.

Since the machine was offline at run time and having it run when the machine come back on line is an absolute must not happen.  Then I would suggest thinking about reaching out to splunk support.