We have a script that splunk executes every minute on the minute...only problem is we do not have this scheduled as an alert or saved search. We cannot figure out why it is executed on the minute every minute. We can see the process in the aplunkd log however we do not see what/why it is being started. We have checked all alerts and no alerts are referencing this script. Anyone have a similar issue? How did you correct?
Well we tried that and it is not finding any references in the splunk files.
If you look in the python log (not splunkd) it should have details on the scripts being called, you can search for runshellscript
Did you customize any scripts/command in splunk?
I have a script running from an alert and it shows in the python logs everytime a specific Webspere JVM is activated or deactived. Sorry this did not help.
We tried that, runshellscript is only in the python logs when a script is run from the cli not when it is called by an alert.
Ok then search for files containing the string that matches your script name. I'm sure there is some built-in GUI tool for searching through the files on disk in windows.
You could move the script out of the /bin/scripts folder. It will not prevent it from being called but would prevent the execution and may generate an error in the splunkd.log to help pinpoint what is calling it.
We have done this, we are looking for a way to stop it from being called.
The server is running on a windows environment. The script should not be executed in Real-time as we have never create any alerts against a real-time search.
Or do you have a real-time search that is still running that calls the script as an alert action?
interesting, can you do a grep -R your_script_name *
in /opt/splunk/etc
and see where it turns up?
It is a script that was created by me, I had it scheduled to an alert. The alert was removed yet for some reason keeps trying to execute the script every minute. I thought it might just be repeating because it did not finish running. I replaced the script with a file that just had "exit" in it, but it is still being run every minute.
what is the script called...? What apps do you have installed?