Search head fails to be added back to the Search Head Cluster after performing a cleanup

[keio_splunk]

After performing "./splunk clean all" on one of the search head, we are having issue to add the search head back to the search head cluster.

Permission error message is reported when adding the search head back to the cluster:

./splunk add shcluster-member -current_member_uri URI:

You (user=admin) do not have permission to perform this operation (requires capability: edit_search_head_clustering).

[keio_splunk · Accepted solution]

The $SPLUNK_HOME/etc/passwd file is deleted after performing a "clean all" on the Splunk search head and there is no administrator credential to access the Splunk instance. We will need to restore back the $SPLUNK_HOME/etc/passwd file by copying it over from an existing search or generate a new passwd file by creating the user-seed.conf. Refer to: https://docs.splunk.com/Documentation/Splunk/latest/Security/Secureyouradminaccount#Create_admin_cre...

Add the search head cluster member after the passwd file has been restored:

:/opt/splunk/bin# ./splunk add shcluster-member -current_member_uri URI: