Having an issue that Splunk doesn't build my knowledge bundles. My setup: One indexer cluster and two standalone search heads (no SH cluster). Both search heads use indexer discovery and the setup used to work fine. Until recently the knowledge bundle of one of the two search heads stopped getting updated on the indexers.
I observe the following:
I did all the usual checks (reboot, filesystem permissions, btool check, ...). On the broken search head, I moved all local apps out of SPLUNK_HOME/etc/apps and emptied SPLUNK_HOME/etc/users and restarted, but the knowledge bundle still wasn't getting build.
In log.cfg on the SH I set DistributedBundleReplicationManager, BundleReplicationProvider, ClassicBundleReplicationProvider, CascadingBundleReplicationProvider, RFSBundleReplicationProvider, RFSManager to DEBUG, but this didn't provide any insights.
Any ideas about where we could search further?