What is the best practice to change the Splunk admin password?

[mlevsh]

We are running multi cluster Splunk environment with a few indexers, few search heads, heavy forwarder, etc.
We need to change Splunk admin password and we will be doing it via command line on each server.
Is there any potential problems we should be looking for?
What else should we pay attention to prior making the change?

Thank you in advance!

[bmacias84 · Accepted solution]

@mlevsh,

There is away of not hard coding user name and password into a script. Its a technique I've used when building servers with Chef (opsCode). Splunk use two files $SPLUNK_HOME/etc/passwd and $SPLUNK_HOME/etc/auth/splunk.secret to secure local accounts.

1. On a fresh install of Splunk update the admin password and create any additional local accounts.
2. Using Chef, SaltStack, Ansible, etc. have the files copied to the new Splunk instance (UF/Enterprise).
3. Create .ui_login in $SPLUNK_HOME/etc this will prevent the UI to prompt you to change the password.

If you would like to take a look of my chef cookbook it can be found here on github. I originally wrote it for Nordstrom for a 5.x installation. It doesnt quite work since I had to remove any Nordstrom specific, but its a good example.

[mlevsh]

@somesoni2, I really hope we don't have any scripts with hard-coded passwords. Thank you for your reply!

[somesoni2]

Are you using any script where the admin password might be hard-coded? Other than that I think you should be good.