LDAP bind password in authentication.conf not portable across hosts?

[mfrost8]

Today I setup another Splunk search head. We use LDAP for user authentication. I copied several files including authentication.conf from our existing Splunk server (combination indexer and web frontend for search). The search head would use the same LDAP configuration as the existing Splunk server.

I found that after I brought the instance up everything was there but LDAP authentication wasn't working. The Authentication Method screen in Splunk Manager told me that my configuration was not returning any groups which wouldn't really make sense given that this is the same configuration from another working Splunk server.

A quick look in splunkd.log showed that it was unable to authenticate. I changed the password used for LDAP authentication in the web interface and everything started working. I would have thought that the encrypted password in authentication.conf was portable across hosts, but it would appear that it is not. Maybe it's encrypted in authentication.conf with the local host name or something? If this is the case, it would be good to have this documented. Is this expected behavior? Maybe it's in the docs and I just didn't see it?

(Also, it seems that the web UI gave me a red herring when it complained that there were no groups being returned rather than telling me that the authentication with the LDAP server failed).

Thanks

[Simeon · Accepted solution]

The problem is that each instance of Splunk will hash the password differently. The way to fix this is to manually change the password in the configuration file and restart the Splunk instance.

[gkanapathy]

It is possible to make hashed passwords match across hosts. The passwords are used using the file $SPLUNK_HOME/etc/auth/splunk.secret. If this file, along with the hash, is copied, then the hash will match.

HOWEVER, if you copy/replace the splunk.secret file on a host, then all other items that are hashed using that file must also match. In a default installation, this will include the SSL key file password hash in $SPLUNK_HOME/etc/system/local/server.conf, and any local passwords in $SPLUNK_HOME/etc/passwd.

Therefore, if you wish to synchronize one of these files across multiple hosts, then all of these files must be synchronized as they all depend on the contents of splunk.secret.

Furthermore, note that if you are using a custom SSL private key for Splunkd, or you have changed the password on the SSL private key file from the default of password and you synchronize the server.conf file as mentioned above, then the password on the private key file in $SPLUNK_HOME/etc/auth/server.pem (or whatever you configure, if you are using a different server certificate) must also match on each server (or be configured differently in the server.conf file).

[ftk]

Simply copy your configuration over and change the hashed password into plaintext. On next start splunk will hash it again.

See this answer for more info: http://answers.splunk.com/questions/1307/i-would-like-to-copy-my-authentication-conf-and-ldap-conf-f...

[Jeremiah · Accepted solution]

I believe this is because each server is using a unique key to encrypt the password string. Check this article out, it might help:

http://www.splunk.com/wiki/Deploy:DeployUsersAndRoles

I've switched to using anonymous bind since Splunk supports it now, you might consider that as well.

[ftk]

Put the password in as plaintext an it will be hashed on next splunk start. IMHO anonymous LDAP lookups are a security risk that can be easily avoided.

[Jeremiah]

I think you're right, but when you use the UI to create authentication.conf, it is created in etc/system/local.

[mfrost8]

I was of the impression that the Splunk 4.0+ way of doing things was to not put stuff in $SPLUNK_HOME/etc/system/local, but rather put configuration components in apps. I was told that putting things in 'local' was the Splunk 3.x way of doing things.

Yes, it does appear that it must be about each server using a unique key to encrypt. Thanks.

[Simeon · Accepted solution]

The problem is that each instance of Splunk will hash the password differently. The way to fix this is to manually change the password in the configuration file and restart the Splunk instance.

[mfrost8]

This works if I change the configuration via the web interface (which will encrypt it and stick the encrypted version into authentication.conf). Are you saying that I could put the unencrypted version of the password into authentication.conf and (maybe after a restart) Splunk will re-encrypt it for me?

Thanks