ERROR CMSlave - failed on move bundle to slave apps err=failed to remove (Permission denied)

[mcronkrite]

ERROR CMSlave - failed on move bundle=<$SPLUNK_HOME$> /splunk/var/run/splunk/cluster/remote-bundle/######-##### to slave apps err=failed to remove dir=<$SPLUNK_HOME$>/splunk/etc/slave-apps.old (Permission denied)

Check what user splunkD process running, is no longer able to manage these folders. Cluster master, and Indexers both need be running with sufficient permissions. Check the permissions on the /splunk/var/run and /splunk/etc/ are readable and writable by the user that splunkd is running as. Find the discrepancy, stop splunk service, and chown recursively the inconsistent directories, restart splunk. If the problem persists, look for another process like a cron job or patch cycle that is resetting permissions.

[mcronkrite · Accepted solution]

Check what user splunkD process running, is no longer able to manage these folders. Cluster master, and Indexers both need be running with sufficient permissions. Check the permissions on the /splunk/var/run and /splunk/etc/ are readable and writable by the user that splunkd is running as. Find the discrepancy, stop splunk service, and chown recursively the inconsistent directories, restart splunk. If the problem persists, look for another process like a cron job or patch cycle that is resetting permissions.