Why is the checksum for active_bundle not matching last_validated_bundle after application?

[fatsug]

This is not a particulary crucial question but it has been nagging me for a while.

When applying changes to indexes.conf on the manager node I usually do a calidate -> show -> apply round. But I am somewhout confused about a detail.

When you validate a bundle after making modifications you get an updated "last_validated_bundle" checksum. In my mind, when you then apply this bundle the now "last_validated_bundle" checksum should become the new "latest_bundle" and "active_bundle". But this is not the case. 

$ splunk apply cluster-bundle
Creating new bundle with checksum=<does_not_match_last_validated_bundle>
Applying new bundle. The peers may restart depending on the configurations in applied bundle.
Please run 'splunk show cluster-bundle-status' for checking the status of the applied bundle.
OK

After the new bundle has been applied active_bundle, latest_bundle and last_validated_bundle checksums all match again. But why is the checksum produced after bundle validation not matching the checksum for the applied bundle?

Have a great weekend!

[sainag_splunk]

@fatsug Hello! The last_validated_bundle  differs from the active_bundlewhich identifies the bundle that was most recently applied and is currently active across the peer nodes.



Refer: https://docs.splunk.com/Documentation/Splunk/9.3.2/Indexer/Updatepeerconfigurations#Use_the_CLI_to_v...

if this Helps,  Please Upvote.

[fatsug]

Thank you @sainag_splunk 

I do understand the "how" so to speak, but I do not understand the "why". If I validate a bundle and then push that bundle I would assume that the checksum for the validated bundle after a push should be the same as the new and current one. Otherwise I cannot validate that the last validated bundle is exactly what has been pushed and applied.

But maybe I'm missing the point? If the checksums are not there to validate changes but rather just to track the presence of unapplied changes, then its fine.

I suspect that at this point the logical follow-up question is, how are bundles modified after validation but before being applied? Or are the checksums only for the changed files/settings and the final cheksum is for the entire bundle?

All the best

[sainag_splunk · Accepted solution]

@fatsug Hi! ASFAIK, the last_validated_bundle is different from the active_bundle because they have different purposes within the bundle lifecycle. The validation process creates a temporary bundle to verify the configuration changes, while the apply process would create a new bundle including additional metadata and runtime information needed for the actual deploy across the cluster. When you run splunk apply cluster-bundle, it creates a new bundle from the master-apps directory and that's why you're seeing a different checksum than your validation step. After successful application, all checksums align since they're now referring to the same deployed bundle.

[fatsug]

While I still find this "counter intuitive" speaking of checksums, I understand that this is by design.

The validated bundle is modified upon application and hence the checksum changes. This is not to prevent pushing bundles that have been modified after validation but rather to keep track of states of changes/bundles.

Thank you for the explanation

[isoutamo]

Here https://conf.splunk.com/files/2017/slides/pushing-configuration-bundles-in-an-indexer-cluster.pdf is an old conf presentation about pushing cluster bundle. It contains quite much information and starting page 41 is troubleshooting section. There are e.g. places where those bundles are stored on CM and peers. Basically you could look their content to see are those different which explains the difference of hash.
r. Ismo

[fatsug]

I feel content accepting that bundle checksums are by design present to visualise the different states and not to detect modifications between validation and application. Thank you for the link though, I will definetly have a look at the presentation. With a little luck it might even be available online somewhere.

All the best

[isoutamo]

This is excellent app for finding old presentations and video sessions https://splunkbase.splunk.com/app/3330

[fatsug]

"There's an app for that" 😄

Thank you!