topic Error trying to push app from deployer in Deployment Architecture

Error trying to push app from deployer

gwobben — Fri, 06 Nov 2015 08:08:43 GMT

I'm getting this error when I try to push an app to my search head cluster:
Error when getting master uri from target to do a rolling-restart Service Unavailable

I've used the command:
sudo ./splunk apply shcluster-bundle -target https://[SOME SEARCH HEAD]:8089 -auth admin:'[THE PASSWORD]'

It stopped working since we've fixed the IP addresses of the Splunk nodes. Anyone an idea how to fix/debug this?

Re: Error trying to push app from deployer

jeffland — Fri, 06 Nov 2015 09:35:35 GMT

I don't know if this causes your problem, but generally, I'd advise against sudoing splunk. It causes all kind of undesired behavior with folder/file ownership and permissions if some splunk actions are performed as a different user.

Re: Error trying to push app from deployer

jkat54 — Fri, 06 Nov 2015 09:46:57 GMT

You need to correct anything wrong with your SHC first.

Run this command from your search head(s) and let us know the output:

 splunk show shcluster-status -auth <username>:<password>

Also it is best practice to create your own splunk user & group, and run splunk as that user.

#If splunk is running as sudo or root, stop it first.
groupadd splunk
useradd splunk -g splunk
chown -Rf splunk. /opt/splunk  #or wherever you have splunk installed

You'll probably have to fix/rebuild the SHC using some of these commands from each search head:

 splunk init shcluster-config -auth <username>:<password> -mgmt_uri <URI>:<management_port> -replication_port <replication_port> -replication_factor <n> -conf_deploy_fetch_url <URL>:<management_port> -secret <security_key>

 splunk restart

http://docs.splunk.com/Documentation/Splunk/6.2.0/DistSearch/ViewSHCstatus
http://docs.splunk.com/Documentation/Splunk/6.2.0/DistSearch/SHCdeploymentoverview

Re: Error trying to push app from deployer

gwobben — Fri, 06 Nov 2015 14:18:19 GMT

We've rebuild the SHC (splunk init shcluster-config ... ) and started the election for captain again. Then things started working again. Thanks guys!