Simple question, has anyone been able to successfully solve this? I can surely think of a bunch of easy ways to accomplish this (i.e. ansible) but what are others experiences? What advice do you have? At this point I have resigned myself to the fact that we have to do it manually, it's not that hard YET. This process is not scalable. I have no doubt that Splunk is working to solve this issue so I dont want to have have our team dev some complicated process around this.
Appetite might be an answer, it's on my list of things to test but if you refer to the slides they presented at conf 2016 or the recording it might be a match for what you are attempting to do.
What isn't scalable? Are you talking about just things like
splunk apply cluster-bundle and
splunk apply shcluster-bundle -target https://shcdeployer:8089? Or is there something more I'm not getting?
And, I would think that those commands done automatically would be like doing a reboot automatically - I'd want to be there when it happened in case something wasn't right.
If I'm wrong, the I want to know the answer to your question, too.
Thanks for the response. We are a global company with sites around the world and due to some countries' laws (Japan, Brazil etc..) data cannot leave the confines of the country. For this reason we were forced to create isolcated splunk instances in those regions. We have a process where we use git to have source control as well as propagate to deployments servers in each region (via S3), which in turn will manage each region centrally. This works awesome, but for each [potentially] small change made I will need to go to each region (which may have multiple SH Clusters) and push the bundle. Hopefully this makes sense.
So you want to automate starting the deploy from one place, which will initiate the deploy the bundles on multiple SH Clusters, but not initiate the deploy on a regularly scheduled time without admin intervention, right?
You can automate the execution of command if you can provide the authentication (
-auth admin:password) in the command itself. Just create a script which invokes those commands. Since in our area we can access deployment servers/deployers/cluster master from one central server (SSH), we are able to schedule a one time job run (using
at command) on all servers at once. If you do regular/scheduled deployment, you can create a crontab job on the server.