Deployment Architecture

Is there an option to automate "restart Splunkd" on the Deployment server?

keishamtcs
Engager

Hi,

I am required to restart Splunk service on deployment clients at mid night everyday .

Selecting "Restart Splunkd" option in the Edit app, i can do it manually.

Is there an option to automate this selection "Restart Splunkd" so that it can be schedule at midnight.

0 Karma

inventsekar
Super Champion

I am required to restart Splunk service on deployment clients at mid night everyday ///

  1. instead going for complex options, i would simply choose the linux's cronjobs - the simple and best solution for these kind of tasks. just write a short script(someone already given above on comments i think)..
  2. configure it to run at midnight, whatever min, hour as per your convenience.
  3. on the script you can include ideas like.. whether splunk restart was successfull or not, get that details and send a mail to you/your team..
  4. All set! ... Happy splunking!
0 Karma

HiroshiSatoh
Champion

It is not Splunk's setting, but it can be restarted from the Splunk server with the following command.

splunk _internal call /services/server/control/restart -method POST -uri https://remoteserver:8089 -auth admin:goodpassword

You can describe this command with SHELL and schedule it with CRON etc.

0 Karma

keishamtcs
Engager

Hi Hiroshi,

This method can be used to restart service on one client. I would like to restart around 15 to 20 nodes.

0 Karma

HiroshiSatoh
Champion

You need to write all the clients that need to be restarted in the shell. I am concerned about the need for ID and password and how to check if the reboot was successful.

0 Karma

sudosplunk
Motivator

Hi,

Are you sure that splunk service is being restarted even if there are no changes to deployment-apps?

AFAIK, forwarder management does have the option to restart a forwarder, but only after a successful installation of a new app or if there is a change in checksum that deployment server is sending.

Reason I ask is, you can automate the process of deploying apps every midnight but this will not necessarily trigger restart if there are no changes to the bundle that is being deployed.

Refer to this link for more info about how deployment updates happen.

0 Karma

keishamtcs
Engager

Hi...

Thanks for the insight. Let me put it this way.

I am looking to restart all splunk services(15 clients) every midnight for a particular app in deployment server.
Using serverclass.conf,restartSplunkd=true is available but this also works if there are any checksum changes.

How do i restart the services using Deployment server or any other method ?

Regards

0 Karma

sudosplunk
Motivator

Using serverclass.conf, restartSplunkd=true will ONLY work if there are changes in the checksum.

If you want to do constant restarts regardless of checksum changes, then you can write a simple shell script and schedule CRON to run the script every midnight. Below is the restart_splunk.sh script I use to restart splunk UF(s). Feel free to modify as needed.

NOTE: Everything below only works for Linux.

### restart_splunk.sh ###
#!bin/bash

## Variables
username=`whoami`
hostname=`hostname`
info='INFO'
error='ERROR'
success='SUCCESS'
fail='FAIL'
logfile='opt/splunkforwarder/etc/apps/restart_splunk_app/scripts/logs/'

echo -e "$(date +%Y-%m-%d:%H:%M:%S) $info $user $hostname $restart msg=\"Initiated Splunk restart\"" >> $logfile

/opt/splunkforwarder/bin/splunk restart --answer-yes

/opt/splunkforwarder/bin/splunk status
if [ $? -eq 0 ];
        then
        echo -e "$(date +%Y-%m-%d:%H:%M:%S) $info $success $user $hostname STATUS msg=\"Splunk is running\"" >> $logfile
        else
        echo -e "$(date +%Y-%m-%d:%H:%M:%S) $error $fail $user $hostname STATUS msg=\"Splunk is not running\"" >> $logfile
fi
  • Create a deployment-app "restart_splunk_app".
  • Create "scripts" directory inside "restart_splunk_app"
  • Place "restart_splunk.sh" in "scripts"
  • Push this app to desired clients using serverclass.
  • Configure the cron schedule on all 15 clients to run the script at midnight.

In Linux, run crontab -e and add this line: 00 00 * * * opt/splunkforwarder/etc/apps/restart_splunk_app/scripts/restart_splunk.sh
HTH!

0 Karma