I've come across an unusual problem while trying automate the installation of the Splunk Add-on for Amazon Web Services. We are currently using ansible pull to execute some scripts which in turn creates a customized copy of the
/opt/splunk/etc/apps/Splunk_TA_aws/local/passwords.conf file. For this we retrieve the credentials via a credstash lookup. This we can do, but restarting the Splunk binary does not encrypt the passwords.conf password values in the Splunk_TA_aws . So we ended up with something like the following:
password = zZ+U..................7HaOS
instead of something like this:
password = $1$B8Ip...........TmHnGo=
Note the $1$ indicating the hash. Security compliance within the organization requires that the secret key be encrypted at rest. However, the only way I've found to hash the password in the passwords.conf file is via the UI by clicking "Configuration" > "Actions" > "Edit" and filling in the secret key then clicking on "Update" within the Splunk_TA_aws. While I can automate this via Selenium Web driver, this adds an additional layer of complexity for an organization that is doing a proof of concept with Splunk and doesn't use Selenium. Is there a Splunk command line tool supplied from the AWS TA that we can execute a shell command to inject the hash into passwords.conf?
I've been bashing my head against this for a few days now and I think I have found the answer. Thanks to Jeremiah's previous response, pointing me to hunt for the right REST endpoint.
curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/Splunk_TA_aws/storage/passwords -d name=Cr4zy4cc355k3y -d password=Cr4zyS3cr3tK3y -d realm=SplunkAWS -d title=SplunkAWS:Cr4zy4cc355k3y:
Proxy config (if required):
curl -k -u admin:changeme https://localhost:8089/servicesNS/nobody/Splunk_TA_aws/storage/passwords -d name=default -d password=:@proxy.server.address.com:3128 -d realm=_aws_proxy -d title=_aws_proxy:default:
This is from the docs:
Manage your accounts, proxy connections, and log levels for the Splunk Add-on for AWS on your data collection node, usually a heavy forwarder, using Splunk Web. Managing these items using the configuration files is not supported.
Also, there's no reference documentation for the config file either. I would guess that (as you suspect) the encryption is happening when the account is created or updated. That means its happening somewhere in the REST API. So a possible option would be to make the REST API call to create the account. At least then you don't have to script something against the UI.