Deployment Architecture

Is there a "What If" command for applying the cluster bundle to let you know whether it will require a restart versus a reload?

Champion

Does anyone know if there is a sort of "what if" option for applying the cluster bundle to let you know whether it will require a restart as opposed to a reload?

I know Splunk has documented which changes cause a restart vs a reload, but I'd rather know for sure what will happen. For example, I had a pending change that just added an additional stanza in transforms and updated props to use that stanza. That was it. But the cluster restarted. So maybe there were other pending changes I didn't know about?

We would like to only apply the bundle off hours if it's going to force a restart, so knowing for sure would be helpful. Anyone know if an option like that exists? If not, I'll submit an enhancement request.

1 Solution

Splunk Employee
Splunk Employee

From 6.6 onwards you can use
./splunk validate cluster-bundle --check-restart

View solution in original post

Splunk Employee
Splunk Employee

From 6.6 onwards you can use
./splunk validate cluster-bundle --check-restart

View solution in original post

Champion

yes, very excited about this...upgrading to 7.0.2 this week

0 Karma

Champion

Thanks for all those that responded, I appreciate the feedback. I opened an enhancement request this morning.

I think it would be helpful if we could have a "what if" option for applying the Indexer Cluster Configuration Bundle.  The goal would be to determine whether applying the bundle would force a rolling restart the cluster or whether the changes would only require a reload.

I know the changes that cause restart vs reload are documented, and that's great.  But there are times that we may have changes staged that we don't know about - poor communication for example.  So knowing for sure whether it will restart would be very helpful.  And since Splunk should know what the current bundle is, what the new bundle will be, and which changes require a restart, then it should seem reasonable that Splunk could tell us what's going to happen.

If we know there will be a restart, then we need to schedule it off hours.  Otherwise, we can do it during work hours.  Right now, it feels like a crap shoot.

I would imagine it to work something like this:

    splunk apply cluster-bundle --what-if

And would result in something like "Applying the bundle will result in a cluster restart"    OR     "Applying the bundle will only require a reload"

Thank you

Explorer

@ maciep - just curious to know if the enhancement request was complete, is there an option available which tells if a restart is required or not?

0 Karma

Champion

hi navanitha, yes they did. It's baked into the bundle validation process. See the following step in the Distribute Bundle section of the docs:

https://docs.splunk.com/Documentation/Splunk/7.1.2/Indexer/Updatepeerconfigurations#3._.28Optional.2...

0 Karma

Explorer

cool, thank you 🙂

0 Karma

Motivator

A prompt saying "this will result in a restart" would be awesome better yet the ability to do a dry run check against the new bundle which should indicate if a restart is required.

Run your check. No restart required. Apply bundle.
Run your check. Restart required. Scheduled to apply after hours.

Please submit that enhancement request! 🙂

Legend

No, there isn't a command option for this when you apply the cluster-bundle, AFAIK.
But if you install the app in a test environment first, it will tell you if Splunk will need to restart. And you should test the app thoroughly before you put it into your indexer cluster.

Also, the cluster should do a rolling-restart, so that not all indexers are offline at the same time. You can set a parameter on the cluster master that determines what percent of indexers will be restarted at once. If you have a search factor of 2, and you don't want a service interruption for users, make sure that the percentage is set so low that only 1 indexer restarts at a time...

Finally, I agree with @dflodstrom - it is always better to push changes off-hours to minimize impact.

Champion

Not to my knowledge.

0 Karma

Builder

Excellent question! We push our changes off-hours for this exact reason.

0 Karma