topic Re: Search Head Clustering SHC - Push config, restart myself? in Splunk Search

Search Head Clustering SHC - Push config, restart myself?

daniel333 — Sun, 08 May 2016 20:40:41 GMT

All,

Is there a way to push a config from a deployer to the search head cluster, then retart the members on my own time?

Background:
We currently have a 24 node search head cluster and are largely happy. Except we are not receiving complaints we restart too often. I was hoping I could restart them one at a time, waiting for the other to come into service before doing the next. A rolling restart so users won't be impacted as much. I see the rolling restart optipn for the SHC. But I don't see an option to push fro the deployer and wait.

Thanks

Re: Search Head Clustering SHC - Push config, restart myself?

jkat54 — Sun, 08 May 2016 23:27:59 GMT

I'm trying to follow you here but it's a bit confusing.

Is it true that you want to do a delayed rolling restart? Perhaps you want to schedule the restart for 1am, etc?

Re: Search Head Clustering SHC - Push config, restart myself?

esix_splunk — Mon, 09 May 2016 03:37:15 GMT

When you push a configuration, it will enforce an automatic rolling restart. This is by design. If you're time constrained on maintenance windows, you should be pushing the configuration and restarting during that time window.

One thing you might consider is adjusting is the percent of SH's that are restarted during the rolling restart. By default, this value is 10% of the total number of SHs in the cluster. You could reduce this to maybe 5%...

http://docs.splunk.com/Documentation/Splunk/6.4.0/DistSearch/RestartSHC#Configure_the_number_of_members_that_restart_simultaneously

Re: Search Head Clustering SHC - Push config, restart myself?

jkat54 — Tue, 10 May 2016 04:31:11 GMT

You can always schedule the restart via crontab for whatever time you like...