Alerting

How to use Splunkweb alert to run a script on the forwarder to restart a service?

max_ruas
Explorer

Hi I have a service that reports to Splunk and some times fell over, is there any chance I can automate this by telling Splunk to run a scripted input on the forwarder level to restart a service, log it and feed the event to Splunk?
Thank you for any answers
Or is there any add on that would do that for Linux like HK Systems Management

0 Karma
1 Solution

skoelpin
SplunkTrust
SplunkTrust

This is probably not the best approach since its a temporary fix. But if you want to proceed then you should do something like this

Put a script on your SH which will SSH to your forwarder then do a Splunk restart. Have an alert trigger this script when your service falls over

View solution in original post

0 Karma

max_ruas
Explorer

I've found a solution on the forwarder app it self. on linux is under
cat /opt/splunkforwarder/bin/scripts/readme.txt
Scripts placed in this directory can be called by Alerts for execution

if you define your scheduled search as an alert, you can configure a script to be run whenever the alert is triggered. For security reasons, the scripts need to be placed in a specific folder like the above.
http://docs.splunk.com/Documentation/Splunk/6.2.2/Alert/Setupalertactions#Run_a_script_for_an_alert_...

0 Karma

skoelpin
SplunkTrust
SplunkTrust

This is probably not the best approach since its a temporary fix. But if you want to proceed then you should do something like this

Put a script on your SH which will SSH to your forwarder then do a Splunk restart. Have an alert trigger this script when your service falls over

0 Karma

max_ruas
Explorer

are you suggesting that in order to run a script on the forwarder I need to do a splunkforwarder restart? can you please clarify your answer?

0 Karma

skoelpin
SplunkTrust
SplunkTrust

No.. What I suggested was creating a script on your SH. Then have that script SSH to your forwarder and restart the service

0 Karma

max_ruas
Explorer

Thanks for the answer I think is valid although Is not what I am looking for as its much more complex that it needs to be, I am looking for some thing like this.. I haven't got around to test yet but I'm hoping it works.
https://answers.splunk.com/answering/232172/view.html

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

.conf25 Global Broadcast: Don’t Miss a Moment

Hello Splunkers, .conf25 is only a click away.  Not able to make it to .conf25 in person? No worries, you can ...

Observe and Secure All Apps with Splunk

 Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

What's New in Splunk Observability - August 2025

What's New We are excited to announce the latest enhancements to Splunk Observability Cloud as well as what is ...