Updating server.conf with Chef, how can I generate pass4SymmKey and sslKeysfilePassword using splunk.secret?

[kdoonan]

I'm trying to keep the server.conf in a consistent state over a few clustered indexes, but I'm having a bit of trouble trying to generating pass4SymmKey (in any of the stanzas) and sslKeysfilePassword (in the sslConfig stanza).

The problem I have is this:

1. Chef updates server.conf with cleartext pass4SymmKey
2. Restart splunk
3. Password is now encrypted
4. Next chef run, the password is updated again to the clear text version and the cycle repeats

I've got around this for now by just not restarting Splunk whenever server.conf is updated, but it seems like a bit of a hack. Ideally I'd be able to generate at least pass4SymmKey so that I'd know if it had changed. Reading some other answers, there has been talk that it is encrypted/decrypted using splunk.secrets, but I've been unable to find out how to do so. Does anyone have any ideas or thoughts on this?

[bevant]

I'm using Ansible, not Chef, but the concept should be the same - and I'm currently dealing with the exact same problem myself. What seems to be working for me so far is:

1. Spin up a Splunk instance somewhere that sets all of your pass4SymmKey stuff to the encrypted version of what you want
2. Take the resulting encrypted password and use it in your recipes/templates/whatever
3. Add $SPLUNK_HOME/etc/auth/splunk.secret to the list of files you manage with Chef, so that all instances of Splunk use the same secret to create passwords.

Now when splunk starts up and checks to see if it can find an encrypted password, it'll pass
I've not tested this for everything yet, but my local dev environment that I'm using to build my Ansible playbooks has an operational cluster using this method - I've only actually tested the theory on the cluster key in the last few minutes, but will get on to the rest of it.

You'll obviously want to protect this secret - I use ansible-vault, but I guess encrypted data bags are probably your version.

It's very early days in my testing, but I can update here with more detail if people are interested - or somebody can update me with the wall they know I'm going to hit but haven't yet 🙂

Cheers.

[kenmaglio]

This is the best approach to having the passwords encrypted and under configuration management. This way if you needed to change the password - you could go to any server, update any conf file with a password (like pass4SymmKey) restart the service, and copy the newly encrypted password back into your configuration management platform. (for us Puppet and eyaml).

[ontkanin]

I'm using Puppet (but similar principles apply to Chef) to do the same thing, and for me the only way around this problem was not setting sslKeysfilePassword in local/server.conf at all.

The default passphrase for sslKeysfilePassword in default/server.conf is password, so this is what my local/server.conf looks like after Puppet deploys my settings:

[sslConfig]
caPath = /etc/pki/tls
caCertFile = cert.pem
sendStrictTransportSecurityHeader = true
sslKeysfile = private/digicert-splunk.pem
sslVersions = tls

As soon as Splunk is restarted, Splunk automatically adds sslKeysfilePassword, and then my local/server.conf looks like this:

[sslConfig]
caPath = /etc/pki/tls
caCertFile = cert.pem
sendStrictTransportSecurityHeader = true
sslKeysfile = private/digicert-splunk.pem
sslKeysfilePassword = $1$APDXHk54jbE6
sslVersions = tls

where $1$APDXHk54jbE6 is an encrypted version of the default passphrase password (or whatever sslKeysfilePassword says in default/server.conf).

So I do not manage sslKeysfilePassword with Puppet at all, and I just make sure that private/digicert-splunk.pem looks something like this:

-----BEGIN CERTIFICATE-----
...
... Server Certificate
...
-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,75422CE10B4307EA

...
... Server Private Key - protected with passphrase 'password'
...
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
...
... (Intermediate) CA Certificate
...
-----END CERTIFICATE-----

Works without a problem.

And you can pretty much use the same approach for pass4SymmKey. The only difference is the default passphrase for pass4SymmKey.
If you take a look at default/server.conf, I believe you'll see it's set to changeme.

I'm sorry, it's not exactly the answer you were looking for, but I hope it helps a bit.

Cheers.

[harsmarvania57]

Hi,

We had same problem with Puppet for sslKeysfilePassword for splunkforwarder automation. But after that we changed puppet code something like, if server.conf is not present on client than puppet will deploy server.conf with plain text password and it will restart splunkforwarder. Once splunkforwarder will restart, plain text password will encrypt by splunk.

Not sure whether you can achieve this by Chef or not.

Thanks,
Harshil

[kdoonan]

Thanks Harshil,

That's pretty much what we're doing at the moment. It works alright until you need to change something in server.conf across everything.

[hamiltonb]

I'd like to know how this is done too!