Hi I have a service that reports to Splunk and some times fell over, is there any chance I can automate this by telling Splunk to run a scripted input on the forwarder level to restart a service, log it and feed the event to Splunk?
Thank you for any answers
Or is there any add on that would do that for Linux like HK Systems Management
This is probably not the best approach since its a temporary fix. But if you want to proceed then you should do something like this
Put a script on your SH which will SSH to your forwarder then do a Splunk restart. Have an alert trigger this script when your service falls over
I've found a solution on the forwarder app it self. on linux is under
cat /opt/splunkforwarder/bin/scripts/readme.txt
Scripts placed in this directory can be called by Alerts for execution
if you define your scheduled search as an alert, you can configure a script to be run whenever the alert is triggered. For security reasons, the scripts need to be placed in a specific folder like the above.
http://docs.splunk.com/Documentation/Splunk/6.2.2/Alert/Setupalertactions#Run_a_script_for_an_alert_...
This is probably not the best approach since its a temporary fix. But if you want to proceed then you should do something like this
Put a script on your SH which will SSH to your forwarder then do a Splunk restart. Have an alert trigger this script when your service falls over
are you suggesting that in order to run a script on the forwarder I need to do a splunkforwarder restart? can you please clarify your answer?
No.. What I suggested was creating a script on your SH. Then have that script SSH to your forwarder and restart the service
Thanks for the answer I think is valid although Is not what I am looking for as its much more complex that it needs to be, I am looking for some thing like this.. I haven't got around to test yet but I'm hoping it works.
https://answers.splunk.com/answering/232172/view.html