I have been running a saved search which triggers a python script for the last few months. I needed to make some changes to the script so I edited it and replaced the version which I kept in splunk/bin/scripts. However since I have replaced the script, it has not been executed when the saved search has run, however the search has run successfully. The following error appears in the _internal logs:
11-03-2014 16:49:36.699 +0000 ERROR SearchScheduler - Error in 'runshellscript' command: Cannot find program 'runshellscript' or script 'runshellscript'., search='runshellscript "myscript.py" "38257" "index=example_search" "index=example_search" "mysavedsearch" "Saved Search [mysavedsearch] always(38257)" "http://localhost.localdomain:8000/app/search/@go?sid=scheduler__admin__search__mysavedsearch_at_1415033340_278" "" "scheduler__admin__search__mysavedsearch_at_1415033340_278" "/opt/splunk/var/run/splunk/dispatch/scheduler__admin__search__gdhtd_at_1415033340_278/results.csv.gz" maxtime="5m"'
runshellscript.py is still in the same place that it always has been - in splunk/etc/apps/search/bin/default. I also made a copy into splunk/etc/apps/search/bin as a I read somewhere that that is where splunk looked for it. This made no difference. Since the script was the only thing that I changed, I tried replacing it with a barebones script that I know has no errors and this still did not work. In fact if I call a script that doesn't even exist, it still come up with the same error. I have also changed all of the permissions so that all the files are read/writeable by anyone. Anybody got any ideas?
So in the end this problem just fixed itself, I have no idea why, but a few days later the script was being called successfully and no more errors!
I have a feeling this is caused by not having the correct splunk "capability" assigned to the user. It is unclear which "capability" is needed - but I'm having this problem and when I add the role "admin" to my user, it starts working fine.
the capability "edit_scripted" is required to use "runshellscript"
not clear if the splunk documenation states this...but via trial and error - it is indeed a fact in v6.2.2